How the Design Journey 2.0 Started

What was wrong

While technology is changing fast, I have been feeling a sense of isolation from the tools and work itself as a UX designer. This seemed like a strange feeling at first, since there were already so many tools to choose from, and there would likely be more in the following days and weeks. It’s also more than possible to change jobs, do a different product or service.

However, it felt like such a long and unnatural slump, that was underlying everything I did. Although I have to say that, this deeper layer of struggle was at first masked under other exploitative factors in the workplace, and rose up into my conscience only recently. In the end, I could not bring myself to find a new product to devote to, and instead spent my remaining saving to jump into a Master’s degree in something digital that’s intentionally vague and open-ended.

I wanted to use the two-year time that I bought for myself, to redesign how to work, from the tools to start with, people to work with, ground up, to the context in which it’s done everyday, for the past decade, and likely for the remaining of my lifetime.

In addition, I’d like to think of myself as just one of thousands and millions of labourers facing the same reality. Some are more well-off than others, and I’m grateful for the people and resources that helped me jumped into the design industry. It gave me a first-hand experience in industrialized, marketing-driven design, to digitally-driven, de-materialized design. It allowed me to see, albeit partially, where we came from: a time of rising material abundance, optimistic globalization; to our current point in time: virtuality and experience over goods, more inequality and relative poverty, more acute realization of unsustainability, all happening in a technology-driven world that have both lights and shadows.

While I was struggling with my relationship with digital products and the working culture generally, generative AI enters the scene that adds a new dimension of challenge. I’m even inclined to say that it has transformed my struggle to something I have never expected before. Since 2022, it suddenly has the potential to change every aspect of ordinary people’s work and life. At current trend, it only seemed that AI will exacerbate the sense of isolation and meaninglessness in modern labour.

So my goal here is to outline my current understanding of why corporate design job has become increasingly unbearable and how AI has added a new twist to it, and to sketch a preliminary action plan for exploring alternative contexts and tools for the future.

How to work/design (in an age of automation)?

Take design and designer as a case in point.

A designer comprehends and thinks through the act of sketching, creating, and talking to (real) users. I would argue that designers can design only through their experiential transformation that happen in themselves through these physical experience.

My first argument is therefore that, a designer should not allow a machine (or another person) to replace them in doing the activities that bring experiential transformation. For example, user research should not be replaced by asking questions to an AI chatbot.

In addition, in order to feel the ownership and be able to do the work, a designer should be able to plan and modify their workflows, as people approach their work differently: some put emphasis on A, and delegate B, while others emphasize B, and automate C. These differences reflect each individual’s intrinsic values toward the work (not market values, although the different intrinsic values would translate into different output, and hence, different market values.)

More than ever, people are required to reflect on the intrinsic value they associate with their work. In other words, what you like and dislike, what you care as a human being (The word, “work”, here, inherits Illich’s definition of work, instead of a common, oftentimes broader definition) .

How AI has affected design work?

AI should act in the same way as creative tools that extend designers’ ability.

For a car designer, in the past, the main tools were: pen, paper, and clay. Sometimes rulers, different brush. Then in my time, there came digital brush, 3D scan, 3D modeling, and real-time rendering. Although I disliked Adobe and would have rather studied using pastel for gradations, here’s an example where technological advancement does not necessarily alienate people: Through digital transformation, some designers could now quickly start from 3D modeling, instead of traditional sketches. The prevalence of 3D modeling tools, such as Rhinoceros, Fusion 360 and Blender provided a new workflow options that were previously unavailable.

These tools share the attributes of:

• allowing easy start
• not requiring complex/dependent processes
• allowing open-ended use cases
• allowing easy modification (even misuse)

So now, how did AI come into the design process ? Here is a table summarizing how AI is used (or plan to be used) in a design workflow, with a subjective comment from a designer’s perspective:

Steps	Process	Use of AI examples	Love/Hate
1	Sketch out quick, dirty thumbnail	Brainstorming, sound-boarding	Comprehensive,　quick❤️ Possibly skewed, lost initial fun, not knowing ones instinctive take☠️
2	Decide on the ones worth exploring further	Generative practices (as in generative art)	Interesting, artistic, post-human approach, expanding ones expressional range❤️ Not suitable for down-to-earth usability design, lost ownership☠️
3	Add details and scenes that clarify envisioned design image	Generating fully detailed image, video, prtotype from start, or prompts to do that	Wow, unexpected, fun❤️ Difficult to iterate, no space of imagination from start, lost build-up process ☠️
4	Add details and scenes that communicate the design to others	Generating fully detailed image, video, prototype, or prompts to do that	Quick yet impressive, facilitate visual creation by non-designers, working prototype by non-coders, less misunderstanding❤️ Difficult to iterate, less interactive process due to eliminated ambiguity☠️
5	Verify the truthfulness of the envisioned design, by developing it closer to the reality (the earlier stage the better)	Full/partial visual/working prototype development through code, explanation, planning, visuals generation	Quick, impressive output, allowing non-developers to develop working forms and mockups❤️ Lost iteration experience that has promoted greater understanding and precision of design, difficult to iterate, lost inner ability to sharpen a vision ☠️
6	Verify if the user can resonate with this vision at different stage of development (the earlier stage the better)	Full/partial visual/working prototype development through code, explanation, planning, visuals generation	Quick, impressive output, allow non-developers to develop working forms and mockups❤️ Lose instant, direct visualization by humans: less ownership on designers’ part, user feedback is filtered through prompts and vector world, lose some iteration process, difficult to iterate☠️
7	Modify the design if the above is not as expected	Iterate with new prompts	Quick, impressive output, allow non-developers to develop working forms and mockups❤️ Lose some iteration process, difficult to iterate, reduced human iteration☠️
8	Design becomes a wholesome decision that reflects the original vision, business feasibility, technical feasibility, and market acceptance	Data analytics, summarize long, multi-modal information points, derive possible strategy options, forecast	Quick, impressive output❤️ Less process value, less initiative and involvement from human partners, the decision can become a Blackbox☠️
9	Keep listening to user feedback, market acceptance	Data gathering and analysis	Quick, easier data-gathering, summery of both quantitative and qualitative data❤️ Replace actual human feelings, hidden (non-verbal, unexpressed) hints with AI generated insight☠️
10	Improve the design, repeating 1~10	Full automation	Quick update, quick fix, efficient use of resources, possibly personalized❤️ Weak in physical, sensory products (lack of embodiment), less designer jobs (more like managerial), Blackbox☠️

Looking across the table, I can’t help but wonder, while efficiency ((quality+quantity)/time) is welcome, how much intrinsic value in the process, the experience, and even the interactions are we prepared to give up in order to keep AI in our workflow?

If we are delegating a part of creation process to AI, then by definition and design, our own process, and hence source of experience, is reduced. However, if we can increase the total volume and richness of our experience through the use of AI, then the above observation can be refuted. In other words, if using AI helps us going further and deeper in our work, which expands our experiential journey and its depth, then the use of AI won’t incur the negativity listed above.

Looking across the Internet, many creators of different backgrounds have already incorporated AI in their process that I’m outlining below:

Vibing

All the process is initiated by AI. The creator may have a vague vision of the end result but never clarified. The creator is in service of the AI to go through and finish the necessary steps of design. The role of AI and the creator is reversed, so that AI becomes a creator, and the creator, a manager.

Vibing may sound fun and worry-free in the initial instance, however, it may also be a frustration-filled experience for people who have some experience in the traditional contents creation. The frustration comes from lots of unguaranteed trials that people sometimes refer to as Gacha (Japanese, as in Gacha Machine). Unsatisfied with a single service, people try across different service providers, spend more time discussing about and comparing different AI generative services, instead of focusing on the act of creation itself.

The practice may not be sustainable in the long run (in terms of personal career, but also environmentally speaking). However, with time (and some effort), people might naturally transition to the following categories.

Sounding board

AI is used to fill in what the creator didn’t think of, other possibilities, a surprising suggestion from another. However, it’s getting increasingly clear that we cannot count on AI to provide us surprising suggestion. In the past, people without previous knowledge expected the unexpectedness, which is only due to low accuracy and hallucination. Some would express their nostalgic feeling toward initial AI models. However it’s questionable how much value we should give to a hallucinating content generated by AI, who doesn’t have a meaningful idea about what it pronounces. It’s value is supported by the active receiver, which is us, the human creators, who can attach meanings to the generated contents and find it inspiring.

Automating

The work process designers feel little attachment to, are increasingly handed over to machines to automate and accelerate, such as translating a format into another, or verifying design in different conditions. However, the evaluation and decision-making role are reserved to the human creator.

We are used to pushing aside our discomfort and performed these tasks because that was what the job required (and, in some ways, they helped fill the long working hours). In an alternative working environment (which will be elaborated later), people can acknowledge their instinctive resistance to work that is clearly not suited for them, identify those tasks, and deliberately seek for help from either machines or other humans.

I would, however, warn against “disliking without trying” attitude, especially in regards to processes that are deeply embedded in someone’s continuous design process. Sometimes, switching off the automation button and doing every tedious, time-consuming process on one’s own can be an immensely satisfying journey. It is important to recognize the socially-constructed, efficiency-driven temporality we have come to accept in our current conditions. But this way of organizing time may not hold in a different world/time.

Expanding

A bit relating to automating, as it still involves changing the already existing design vision into another relevant format, but in a way bridging the gap between the design domain and other specialists.

In cases where non-coders can change design visuals to working prototypes with vibe-coding, the designer has the potential to think from a working prototype instead of traditional static sketch. With careful observation and trials, the designer is exposed to more coding and development world. The experience is hence augmented and the designer empowered.

Some also emphasized the usefulness of AI in terms of lowering the floor of new knowledge domains. Mediated by AI, people no longer need to start from 0 and build-up their knowledge reading the whole book, or following the entire course. Instead, people with urgent, pin-point needs can use AI to extract and adapt the knowledge in a digestible format for them to consume easily.

Experimenting

I believe AI is but another creative material that has its own personality (albeit immensely more powerful). A designer can experiment with brush stroke the same way as experimenting with AI generated results. The interest hereafter lies in AI itself. A creator of this category may as well be identified as an AI creator, rather than their original role.

The interaction with the AI is driven by the curiosity. The designer proposes an experimental stance: “asking whether it might be possible to talk about “AI” by focusing on what is still unknown”, as proposed by an artist from Japan.

岸裕真 https://wired.jp/article/yuma-kishi-interview/ https://prtimes.jp/main/html/rd/p/000001721.000012109.html http://ccbt.rekibun.or.jp/art-incubation/40334 https://it-hihyou.com/recommended/49910/

What matters now is how we might welcome “AI” as something fundamentally different from humans. Can we approach “AI” not as a substitute for humans, nor as an earthly overlord that takes our place, but rather as a new kind of “unknown” whose shape does not fit the human mold?

This book was written as an experiment to explore whether we can receive AI in that way—not by following the conventional frameworks that define our relationship with AI, but by intentionally stepping outside them to consider an encounter with this “unknown.” Excerpt from 『未知との創造』, translated by the author

Agency is the key

If there are ten designers, there are likely more than ten different ways of using AI. In an optimistic future, we can expect an exponential growth of use cases—ones that will emerge only when we are able to situate this new technology within a meaningful cultural context. I have little objection to anyone using AI (setting aside the environmental cost, which is a significant concern for me), SO LONG AS THE PERSON’S AGENCY IS NOT UNDERMINED.

When I say human agency, I’m not thinking of it as something independent from the outer world, and especially the technology surrounding us. We are transformed by technology (we are Cyborgs). When I say agency, it is the state in which a person is actively receiving, being transformed, but then is capable of actively changing what they are receiving, to a direction they desire.

If we try to come up with a composition diagram of our Cyborg body, just like the ones that we have for our human body (65% water, 20% protein, 12% lipids..), I’m made up of tools that I’m using, other life forms that my life depends on (my gut flora, the meat I ate, my pet), and people and ideas that changed who I am.

This new composition opens up our mind to a different societal context that could be possible. Each individual, me, you, Jay across the hall, we all potentially possess a spectrum of different compositions respectively. Furthermore, each individual can potentially have any numbers of these spectrums, depending on the numbers of roles/identities one is open to in this world. So the me_designer might consists of 60% flesh/blood, while me_marketer might be 60% digi. It directly reflects my world view.

When all the identities are added up into one human actor, we want to make sure the person is maximizing the potential, satisfaction, and enjoyment as a human being. When the tool in this case is AI, we can start thinking about and visualizing our relationship with AI in multiple aspects of our life.

Moreover, I want to emphasize that the roles that I refer to is not limited to professions. It includes any role that we undertake across our life time: a student, a parent, a caregiver, a resident. Current social context values professions that have economic values, but historically devalued other activities that were difficult to attach market values, but nonetheless with unreplaceable intrinsic values. If we try to think about a different possible social context, it’s important to recognize this bias in the first place. In a world where the environment suffers from our uncontrolled economic greed, it’s exactly these undervalued activities such as care that uphold the valuable insights in how we should redefine our relationship with work. And then we will realize that, some work has no room for machine/digi, while others have abundant.

Going beyond the individual, people with similar social roles in the society as a collective, also has an ideal balance between AI (tool) and human input. Since as a practitioner of a similar activity, there is more often a common language and expectation. The collective composition may be plotted close to the average of aggregated preferences.

The question then falls down to, in our case, what is the preferred collective composition that applies to our current society. And more importantly, how can we measure the personal preference of each individual? In order to do that, the most crucial use cases to define are around the edges of tool-dominance meeting human-dominance. One way of telling a person’s preference in a specific domain, is to offer the edge use case where tool dominance is increased step-by-step, until a point where, passing that point is no longer desirable for the person in question.

Bringing contextual change to work

HCI designers have the ability to make sure that the tools are empowering people, that they stay in the limits within the first and second threshold as described by Illich. The key is the design of the tool, instead of the technology behind it. Technology can be as “powerful” as it could, but the design needs to make sure it is empowering people, instead of controlling them.

At the same time, designers have a long history of serving the Capitalism. Like the designer in Mad Man. A car designer follows business-driven cycles of minor and full model change and works out new designs that will stimulate more and more demand, ultimately for the profit of the company.

It’s high time designers realize how much we are incorporated into the Capitalist system that’s perpetuating the user-designer division. At the same time, I’m speaking for myself but hopefully also for many fellow designers, that by doing so, we are also distanced from our own design, that we are but a tool for more clicks, more scrolls, and more profitability.

Creating a tool that empowers the users, will in turn empower the designers. But a tool that empowers the users does not necessarily makes money and drives business expansion. It is likely to be shared and contributed by the users, and it embraces user autonomy, instead of dependence.

I think the problem of Capitalism can only be overcome internally. Users vote through purchase and usage. If there are common digital tools that are open and free for anyone, that are better than centrally managed, charged options, then the market will eliminate the latter, until the time comes, when new business opportunity is found by another company.

So continuous improvement and evolution are necessary to maintain the digital tools in good condition. In order to do so, there needs to be a social structure where designers can be involved in these roles continuously.

Freelancing offers a way to earn a living while working with profit-driven companies, gaining firsthand exposure to the advanced technologies and products they develop to shape markets and influence behavior. At the same time, it provides an opportunity to redirect some of that knowledge, experience, or benefit back into the commons.

It is one of the working style I can think of that can reduce inequality, and enhance sustainability. However, does that mean that the designers need to voluntarily give up some of their earning for the Pro Bono project?

Even the Linux Foundation relies on profit-driven companies to be members and offer resources in order to start even a sandbox project. There is politics of give-and-take at work here. Especially because open source can be a great source of innovation. Thanks to many decades of engineers, scientist, and designers believing in the democratization of the Internet, the digital fabrication, the data, and the now the AI models and agents, the open-source communities have strong ground, connections and appeals already. And a contribution of a single person can be small, but it is the power of the community to be able to multiply that by thousands and millions of contributors to create something greater than any one organization can achieve. Think of Wikipedia, it didn’t exist before.

The most important is to expand the base population who can contribute, no matter how small, but constantly, and with positive sentiment of ownership and care.

Concluding remarks

Recently, an AI company CEO suggested, as more and more work is effectuated by AI, they started to have a different perspective. A task in which humans play an active role, even if some parts are delegated to AI, can be validated or modified at later steps. However, when a task has no more active involvement of humans, it incurs costs to the company in the middle- and long-run as nobody on the team has neither the initiative nor the knowledge to undertake the black-boxed task again. So even profit-driven companies would agree with my approach of empowering people at least to a certain extent. But beyond that point of agreement, how should we reconciliate?

If the company owners are only interested in “making money” button, then once there is an AI that can reflect and correct their course of action, and provide results that surpass anything done by human, then is there any reason to keep any human employees?

It is worrisome that, I’ve met (powerful) people who genuinely believe in the scenario where, the “winners” of AI will get to keep or, enlarge their privilege, while “losers”, most of the rest of the people, have neither the need nor chance to work, are given a universal wage, and cheap AI-generated contents to entertain themselves with, and that is how everyone can live happily ever-after.

It is time for people to refute that future.