Generative Experience Design

By Chris Duffey, Sr. Strategic Development Manager, Adobe

Chris Duffey, Sr. Strategic Development Manager, Adobe
Chris Duffey, Sr. Strategic Development Manager, Adobe
Let’s go on a journey and explore the evolution of creativity and design for the new world retail technology order. Creating future brand success, growth, and customer value hinges on companies’ ability for content velocity: the ability to create, source, manage, and publish content faster and more effectively than ever. We are betting big and predicting that recent cutting-edge technology advancements in Generative Experience Design will empower you to thrive in this rapidly changing world.

My first car growing up was a third-generation hand me down of a 1986 Chevy Caprice Classic. I loved that car, it was an absolute boat, but I could pile in all my friends and we had a blast with it. One of my favorite features was it had power seats and power windows.

These features always reminded me of one of my favorite commercials at the time, the classic Grey Poupon commercial. Created back in the 80’s by an IPG agency in which a Rolls-Royce pulls up alongside another Rolls-Royce, and a passenger in one asks “Pardon me, would you have any Grey Poupon?” The other responds, “But of course!” Before the commercial Grey Poupon had only a 2 percent share of a mustard market and after the commercial they tripled their sales. Showing the power of creativity – It was one of the reasons I got into advertising, design and technology.

But as much as I loved that caprice classic, I had a love hate relationship with it, it would guzzle my weekly allowance in gas. Even then I thought there had to be a more energy efficient way. Over the years the car industry has transformed from the Combustion Engine to Hybrid to Electric and what they learned along the way was for true transformation to take place an industry can’t just go in half-way! They have to go all in to truly innovate. And by going all in, electric cars are and will continue to have a profound impact on business, the environment and the world as a whole.

And much like the car industry we as a creative/technology industry must not only continue to evolve and transform “But we too must go all in” by merging creativity, data and technology to enable us to achieve a holy grail of sorts – personalization at scale. To ultimately create deeper more engaging 1:1 connection with customers.

Part of the automotive industries disruption is not only coming from leveraging data and technology for the electric engine. But also, how technology and data is informing their design. Take for instance Tesla’s new cyber truck. While Cyber truck’s design is aesthetically a shocking departure from a conventional pickup design. There are actually many benefits to its data and technology informed triangle design. Improved Performance – The triangle design improves the center of mass, creating a more balanced vehicle. Improved Safety – The triangle design allows the passengers to be centered closer to the midpoint of the car. And by doing so protects passengers from both forward and rear collisions. The triangular design will increase the ease of manufacturing.

As an interesting aside, the power of the triangle can even be traced back to Nicola Tesla himself. Which by the way, Tesla is responsible for 80% of the energy technology we still use today. He believed the triangle was a harmonizing creation. A universal constant and that everything in the universe obeyed the law of the 3-sided triangle, for example not only the shape of the Pyramids of Giza but also the location itself.

Much like the triangle and the power of 3, now with advancements in technology the retail industry has the capability to power infinite unique experiences. We call this framework Generative Experience Design. Comprised of 3 elements: Human Computer Interaction Design, Generative Design and Data-Driven Design

To put that in perspective, let’s say we have a triangle, and then we take another triangle and place it on each of the 3 sides of the triangle. We get a star and we keep doing that for every one of the sides, we could do that for ever and ever. This is called a Koch snowflake and is an example of a fractal. And much like this we can create infinite unique snowflake experiences at scale by combining Human Computer Interaction Design, Generative Design and Data-Driven Design.

Human Computer Interaction Design
The first aspect of Generative Experience Design is Human Computer Interaction Design HCI focuses on the interfaces between people (users) and computers. And how the future of technology can transform the experience humans have with machines. HCI surfaced in the 1980s with the advent of personal computing. As technology continues to become part of our daily lives, let’s look at the importance of 5 criteria as part of the design process:

1. Ease of Use: There are 3 measurements to consider: Learnability, Efficiency and Error tolerance. Learnability – “How easy is it to learn how to use the product. Efficiency – how quickly are users are able to perform tasks. Error tolerance – An interface is considered error tolerant if it prevents potential user errors.

2. Pleasantness: Humans are drawn to beautiful things and make quick judgements when seeing an interface for the first time. Amber Case in her book Principles and Patterns for Non-Intrusive Design talks about how can you design technology that becomes a part of a user’s life and not a distraction from it? This concept of calm technology has never been more important.

3. Safety: The safety of an interface is an important consideration. Whether there is potential for users to do harm as a result of engaging with a specific action. UI must not be confusing and designed for clarity with safety in mind.

4. Security: Designed so the user can be able to securely send and receive information. Things like warning messages to the user to be aware of possible dangers.

5. Accessibility: Making technology usable for all abilities including those with physical, visual, speech, auditory neurological and cognitive disabilities.

Applying these HCI principles are the foundation for powering infinite unique experiences.

Generative Design
Next, let’s discuss the opportunities when you combine HCI fundamentals with recent technology advances in Generative Design. This combination will leapfrog the singular notion of just a data driven design approach. What we mean by that is now we can collect information such as brand, channel performance of content and customer profile information. And then generate content and feed it back into a multi-variant testing framework. From there we keep generating and integrating, and through machine learning systems make recommendations to get tighter and tighter with each recommendation.
Generative Design is being driven primarily by two technologies. The first one being GANs and a subset called CANS. GANS stand for Generative Adversarial Network. A type of machine learning that was developed in 2014. Essentially, two neural networks compete with each other in a bit of game – in the sense of game theory.
When given a training set, this technique learns to generate new data based off the stats of the training set. For example, a GAN trained on photographs can generate new photographs that look at least superficially authentic to the human eye, having many realistic characteristics.

In 2018 the possibilities of GANs gained a lot of notoriety with a painting called Edmond de Belamy

Created by Obvious – a collective of researchers and artists. They worked with the latest models at the time of deep learning to explore the creative potential of GANS. The algorithm referenced 15,000 portraits from various periods. The AI generated portrait was sold at Christie’s auction in New York for $432,000. Underscoring the important direction Generative design is going.

From there GANS have quickly advanced. Take for example “This Person Does Not Exist”. Which allows you to add computer generated photos of fictional people to your design mock-ups. This Adobe XD plugin uses technology in partnership with Nvidia as a source for photos and images.

Another example of GANs let’s online shoppers virtually try on clothes. Adobe developed this in partnership with Stanford university. This version of Deformable GANS – fits the garment to the shape and pose of a customer’s image. Trained on 19,000 images of models and product listings – to understand the nuances of composition and textures. Allowing new ways shoppers can try on cloths virtually has taken on even more value recently.

The second technology that is accelerating generative design is Natural Language Processing (NLP). Essentially, NLP is a technology used to aid computers to understand human natural language NLP is a driving force behind generative experience design. Syntactic and semantic analysis are a couple main techniques used to complete NLP tasks to power features such as:
• Text generation
• Sentiment analysis
• Part of speech tagging
• Language translator
• Visual content understanding
• Q&A makers
• Speech to text/text to speech

We already have features like text summarization to fit different mediums, text analysis on the effectiveness of the headline and then generation of alternate headlines for you to get started.
In terms of the future of speech, the ability to map pixels to audio is quite exciting. Associating the right image with voice, essentially relating with what you see with what you hear and then clustering the data of the words associated around the pictures for added suggestions.
Data-Driven Design
Design that is backed by data and helps understand the target audience better, is known as data-driven design. Data can prove that your work is on the right track, reveal the users’ pain points and opportunities while unearthing new trends. As well as improve your designs by adding objectivity to them. But how do you practically use data-driven design? An Adobe XD to AEM to AEP workflow is an example of how designers can work closer with data and developers to create content at scale. In order to move from Design to Live Experience, with faster orchestration.
Step 1: Adobe XD is the collaborative platform for experience designers to design and prototype engaging user experiences. The first step of Data-Driven Design is utilizing Design Systems. By utilizing Design Systems in XD, allows for breaking down designs into smaller reusable elements. Think of it like building blocks, smaller level elements that can be built upon one another. Which can then be placed into templates and pages. And this is all based on the idea of reusability. This means structuring design deliverables as a design system – you will have a centralized collection of reusable components of data, accompanied with usage assets. And by using a design system creates a shared language to enable a consistent experience across all devices.


Step 2: Adobe Experience Manager (AEM), allows retailers to get personalized, content-led experiences into market faster. AEM combines digital asset management with the power of a content management system. Within an AEM instance are core components, inclusive of UI and code. XD Design System will map to these core components. By building an XD Design System with AEM’s Core Components in mind. This library of building blocks (both UI + code) there will be less coding involved, and the ability to launch personalized sites will be faster.

Core components’ open source maintain set of production ready elements. These include flexible configuration options right out of the box. So, mapping your design to use core components you are already saving precious development time. These core systems use building blocks such as buttons, titles, accordions, build responses, mobile friendly and accessibility compliant global standards. These AEM UI kits make it faster than ever to design and develop new digital experiences.

Step 3: Adobe Experience Platform helps you deliver the right experience every time by transforming all your data into robust customer profiles that update in real time and AI-driven insights you can activate in every channel to:
• Stream data in
• Prepare it
• Put it to use
• Activate anywhere
Generative Experience Design
In the end it is about serving the creators and marketers. Freeing them to focus on what matters – design and delivery of breakthrough experiences that engage and delight the consumer. And by also respecting the ethical implications of the technology – respecting the privacy of the customer and their data.

Now with the Generative Experience Design we will be able to power infinite unique experiences. When grounded with HCI Design, Generative Design and Data Driven Design. Yet the more things seem change the more they stay the same: It all starts and ends with the customer.

We believe Generative Experience Design will amplify human creativity and intelligence, not replace it. Empowering people to create, optimize and deliver next-generation experiences in completely new ways. Looking around the corner at “what’s next”, Innovation will continue to be the lifeblood of the retail industry.