The Design Behind It

a notebook from Organization Design Studio®

In Mumbai, a store manager is changing the restocking process to reduce the time it takes each night. An IT team leader at a construction company in Paris is incorporating AI into how coding is done for internal management tools. A plant manager in Toronto is developing a new supplier assessment process and criteria.

Not one of them has "designer" on their business card. All three are designing, reshaping how work gets done, what technology is used, and who decides what.

In this issue of The Design Behind It, we will address what these designers have in common, how to keep AI from narrowing your options, and why better numbers do not settle an argument.

Who Designs Your Organization?

While there are professional organization designers, far more leaders design, regardless of title, seniority, or function.

Organization design work is cross-functional and multidisciplinary by nature. It takes a wide view of the whole system and deep knowledge of the part being rebuilt. Designers carry both at once, which is the hard part.

Organization designers share eight characteristics, organized into four domains: how designers think, why they act, how they interact, and how they learn.

Cognitive: openness to experience, systems thinking

Organization designers think differently than many managers. Instead of learning to optimize the current design, they are curious and explore alternate possibilities. They are open to new ideas and view the organization as a system of interdependent activities rather than independent functional silos.

Motivational: never satisfied, persistent

What drives designers to act? Motivational traits determine whether a designer will push for improvement, persist in the face of resistance, and uphold high standards. These attitudes shape how a designer shows up in their work, especially when it is difficult.

Social: collaboration, empathy

While many creative pursuits are individual activities, organization design requires diverse perspectives across multiple domains. Consequently, it is a social process of collaboration. Effective design teams build trust, understand and integrate diverse perspectives, and co-create new solutions.

Learning: evidence orientation, learning from the past

Designers learn from empirical evidence what works, what doesn't, and under what conditions to design better organizations for today and the future. They are lifelong learners who think deeply and utilize historical performance trends and experiences to inform their decisions and drive effective strategies.

The tell is what someone does with the design they inherited. Most of us learn to optimize what we were handed and get good at it. Designers are not satisfied with the status quo and refuse to settle for "good enough." This is not restlessness, but purpose-driven ambition.

Good news: most of the eight are mindsets and habits, not credentials. The mindsets can be adopted, and the habits can be practiced. What will you do?

Read the full article

The Bench

Many organization design teams are composed of the people who do the work. They are not full-time designers and, in many cases, have spent their careers optimizing the systems they were given. Incorporating AI to expand access to new ideas and novel options boosts the creativity of the team members who come to the project with less developed design instincts. This edition of The Bench addresses how to get the benefits of AI and mitigate the limitations.

When you shift from polishing organizational designs with AI to uncovering options that your team unconsciously filtered out, the number of diverse ideas increases. Diverse ideas enhance the development of new and innovative designs. The trick is to get the AI to produce multiple starting points, creating multiple branches of possibilities to explore.

Getting started

Here are five practices inspired by Doshi and Hauser (2024) to get started using AI to uncover options.

Pre-empt Cognitive Anchoring: Have the design team develop two to three options before they open the AI tool. This will help avoid that natural tendency to anchor to AI-generated starting points. While there are situations where we don't know where to start, that is seldom true with organization design issues that people struggle with every day.

Multi-Logic Prompting: Have the design team vary the AI brief instructions, including cost efficiency, risk mitigation, lead times, life cycle sustainability, etc. This will help force the model to build multiple distinct designs that can be further explored and evaluated.

Require Explicit Trade-offs: When your team develops the options, have them require the model to identify three key elements for each option.

1. What the design optimizes: what are the primary gains?
2. What the design sacrifices: what are the hidden costs of the option?
3. Where the design breaks: what are the failure conditions with this design?

Retain Criteria Ownership: While you may find it useful to have the AI evaluate its own work in a new chat, avoid letting the AI define the evaluation criteria. Judging what "good" looks like is work for the human team members. Remember, the AI can't be held accountable; only humans can.

Beware of Team Self-Evaluation Blind Spots: While reflection and self-assessment are useful practices for any design team, by themselves, they are unreliable indicators of quality outputs. The antidote is to incorporate third-party (internal and/or external) evaluation into your design review process. This approach can also help expose blind spots, identifying designs that the team would not have considered due to institutional inertia or their own cognitive bias.

Homogenization

While Doshi and Hauser (2024) found that third-party evaluators rated AI-assisted stories as better written and more enjoyable, those same stories were measurably more similar to one another than stories written without AI. The corollary for our organizations is that each individual team leader is incentivized to produce high-quality design outputs using AI, but the organization as a whole can suffer from a narrower set of systems and design options. Taking this one step further, if every organization uses similar models and prompts to design their systems, competitive advantage declines through structural convergence.

Good news! You do not have to choose between leveraging AI speed and preserving organizational uniqueness. Write your own options first.

The objective: A human-driven approach to organization design that leverages AI assistance while mitigating the downsides.

The Evidence

Designing for the wide variety of complex humans involved in an organizational system is challenging. Getting the desired behavior from our designs can be frustrating, particularly when the humans involved do not always interpret the instructions and data the way we intended.

What is Motivated Numeracy?

A seminal study by Kahan et al. (2017) found that human interpretation of data is seldom neutral. When presented with data related to politically charged topics (e.g., gun control), people often misread the numbers to align with their ideological worldview. When presented with the exact same data but related to a neutral topic like skin rash cream, they read the numbers accurately. Ideological identity distorts how people read data, and Kahan's findings on this aspect are confirmed by Persson et al. (2021). But that is only half the story.

The controversial part of Kahan et al. (2017) was that higher numeracy did not correct the bias. In fact, they argued that quantitatively skilled participants were the most polarized, selectively using their analytical skills to defend their identity. This is distressing to many who have proposed that the answer to misinterpretation and bias is more statistical literacy to counter an information deficit. When Kahan's initial working paper came out in 2013, Marty Kaplan characterized the study as the "most depressing brain finding ever" published in an article by the same name (Kaplan, 2013). However, the same replication study cited above that confirmed the ideological identity finding did not find strong evidence that higher numeracy made the bias worse (Persson et al., 2021). Whether being good at numbers makes the bias worse remains a contested issue and thus one that cannot be relied upon when designing organizational systems.

Caution: This can also be a dangerous finding if the designer believes that numeracy amplifies bias; then they might conclude that investing in data and analysis capability is counterproductive. That would be a mistake. Analytics capability is not just technology; Akter et al. (2016) model it as technology, management, and talent together, with payoff depending on how well it aligns with strategy.

Implications for Organization Designers

As organization designers, why do we care?

First, we can't assume that designs that deliver better data will produce objective, evidence-based decisions. Second, we can't assume that staffing our systems with stronger quantitative talent will make them more objective in all decisions. The practical conclusion for designers is still that high-quality data and analysis, combined with quantitatively talented people, are essential for good management, just not sufficient in and of themselves. Good designs need more than that. The capability is also buildable; Peters et al. (2017) found that adult numeracy performance is malleable, with measurably better decision outcomes.

Our objectivity is influenced in many ways, some we don't even realize. We come to work with our identities, our values, our experiences, all of which impact our interpretation of the information we see and hear. In all organizations, metrics are evaluated by people who have a stake in the result: a budget, headcount, reputation, or a project to get funded. Consequently, we need to design organizational systems for the reality that individuals will interpret our designs differently and thus behave in different ways.

While designing accurate data and information, along with analytical and visualization tools, is critical, we also need to design the environment people are in when the data arrives.

Three ideas to get started:

Fix the criteria for decisions in advance. Establishing success metrics and operational thresholds in advance of the analysis helps prevent stakeholders from shifting the "goalposts" after seeing the results.

Design the system to include outsiders (at least one) in analysis and major decisions. Involving people who don't have a personal, financial, or political stake in the outcome adds an objective input to the process.

Develop hypotheses in advance as part of the systematic process. Require leaders and analysts to formally write down their expected results before they query the data or model.

From the Studio

What we're building, publishing, and teaching.

The Labs are getting close. Behind the scenes at the Studio, each Lab is being built, tested, and refined against our standards for depth, rigor, and usability. At launch, you will have the complete design lifecycle in one place, from design brief to discovery to diagnosis, design, development, and deployment.

References

Akter, S., Wamba, S. F., Gunasekaran, A., Dubey, R., & Childe, S. J. (2016). How to improve firm performance using big data analytics capability and business strategy alignment? International Journal of Production Economics, 182, 113–131. https://doi.org/10.1016/j.ijpe.2016.08.018

Doshi, A. R., & Hauser, O. P. (2024). Generative AI enhances individual creativity but reduces the collective diversity of novel content. Science Advances, 10(28), eadn5290. https://doi.org/10.1126/sciadv.adn5290

Kahan, D. M., Peters, E., Dawson, E. C., & Slovic, P. (2017). Motivated numeracy and enlightened self-government. Behavioural Public Policy, 1(1), 54–86. https://doi.org/10.1017/bpp.2016.2

Kaplan, M. (2013, September 16). Most depressing brain finding ever. The Huffington Post. https://www.huffpost.com/entry/most-depressing-brain-fin_b_3932273

Persson, E., Andersson, D., Koppel, L., Västfjäll, D., & Tinghög, G. (2021). A preregistered replication of motivated numeracy. Cognition, 214, 104768. https://doi.org/10.1016/j.cognition.2021.104768

Peters, E., Shoots-Reinhard, B., Tompkins, M. K., Schley, D., Meilleur, L., Sinayev, A., Tusler, M., Wagner, L., & Crocker, J. (2017). Improving numeracy through values affirmation enhances decision and STEM outcomes. PLoS ONE, 12(7), e0180674. https://doi.org/10.1371/journal.pone.0180674

— John