Gen UI on Watch - Peiqi Tang

Manual configuration can't keep up with the ever changing life

The endless scenarios we run into require endless watch faces that we cannot produce in the traditional way.

Three iPhone screens showing manual Apple Watch face configuration for color, complications, and complication choices.

Context based generated watch faces throughout your day

Taking into your past and current context, an AI agent has the potential to generate in real time a watch face that brings you the right info at the right time. So I built a prototype webpage that allows users to enter simple context, hit a button and get an AI generated watch face.

Go to GitHub Repo

A dance on the agency spectrum

In this project, in order to help AI decide what to show on the watch face and how to design the UI, I created 3 types of local files for AI to reference.

Broader context

Because the prototype is in a "sandbox" environment, I created synthetic context for the model to understand what's going on broadly.

Content types

Categorize the type of content we can show on the watch, and their related metadata.

Design guidelines

Include atomic components, font styles, colors and instructions on layouts.

Pipeline diagram showing current context entering a webpage simulator, an AI model decision layer, local files for broader context, content types, and design guidelines, then outputting a rendered watch face.

How much guidance should I give the AI model when it creates a watch face? If I force it to choose from one of the provided templates, the template list has to be very inclusive and exhaustive, which is hard to achieve and even harder to maintain long term. But if I give the AI too much freedom, it will mis-use the components and create layouts that don't look good.

Agency spectrum diagram showing stricter templates on one end, full agency on the other, and watch face layout in the middle.

I thought I would position my project somewhere in the middle, maybe slightly towards more freedom and agency. However, I figured out the AI's creative capabilities are not evenly distributed across different aspects of the UI. AI can perform well on some decisions while struggling on others.

Takeaways

It takes practice to find the best way to construct layout guidelines to agents. And the learnings may not sustain another model upgrade.

Creating what makes you YOU still takes time even with more powerful AIs in the future.

Make your design guideline/system agent friendly.

People will always want some stability and predictability on UIs.