Machines and Society

As visual designers, integrating ChatGPT and other A.I. technologies into the visual design workflow has the potential to revolutionize the way we approach our work. 

A solid process of visual design involves identifying and defining a problem, collecting and analyzing data, testing and iteration, and creativity and innovation, which is just like any scientific research. By utilizing machine learning algorithms for content creation and analysis, designers are equipped with new ways to brainstorm, lower costs for visual communication, expanded options of tools, which will push our design to the next level. Such innovation could also help us to rebuild our visual design workflow, as we seek to incorporate new tools and strategies to optimize our design processes. The possibilities are endless. 

Below we illustrate three directions for incorporating A.I. assistants like ChatGPT into the visual design workflow. 

The goal of the research phase is to gather as much information as possible to inform the design process and ensure that the resulting design meets the needs of its users and aligns with the goals of the project.

ChatGPT is a good companion to work with in brainstorming or conceptualization phases. It is very good at chatting, and it is patient, logical and knowledgeable. Designers can ask ChatGPT to generate ideas to:

• Provide preliminary guides and structures to an unfamiliar field
• Gather project requirements and goals
• Explore ideas and gathering inspiration
• Dig into a topic
• Fill in details

Quick Tips:

• Be aware of facts, data or links ChatGPT provides, and always verify their sources. 
• If the answer you get is not desirable, let ChatGPT know. Generate it again, or let ChatGPT provide ten more, or rephrase the question entirely. The answers can be different in terms of content or even approaches. 
• Better questions yield better answers. Here are some ways to improve the questions: including context, providing examples, and specifying the direction.

While ChatGPT is great at generating text, it cannot generate images (yet, as of writing). But it is capable of facilitating sketching by combining ChatGPT with A.I. image generators. 

If you are already using A.I. image generating tools, such as DALL·E2

• ChatGPT can improve the expression of your prompt for image generators to understand your descriptions better;
• ChatGPT can elaborate or visualize a blurry idea;
• ChatGPT can easily make hundreds of variations to expand your options.

In summary, we can use ChatGPT to provide or enhance image descriptions as a new method for creating sketches or visual ideas. 

Most revolutionary to the production phase is that visual designers can now leverage the coding capabilities of ChatGPT and feed that into any software applications that have a coding interface. In short, ChatGPT can serve as a valuable technical helper for designers by providing scripts with personalized and detailed instructions. 

Here are some examples where ChatGPT can come to help:

• You can utilize code without being an expert in coding languages;
• You can more easily make tedious adjustments through programming;
• You can customize new features/plugins for software.

In the section below, we list some mainstream applications that are able to utilize ChatGPT through code. 

(1) Adobe After Effects | Motion graphic software

• Expression, a small piece of JavaScript code that you can put into animated properties in your After Effects projects.
• Script for making your own plugin.

(2) Blender | 3D software

• Add-on, a Python module with some additional requirements so Blender can display it in a list with useful information.
• Python Scripting for rendering that offers full programmable line styles.
• There is a ready-made add-on, GPT-4 Add-on for Blender, for using natural language to create models and visuals.

(3) Processing | Flexible software sketchbook
Processing supports multiple coding languages including Java, JavaScript, Python, and Ruby. It works on an arts-oriented approach to learning, teaching, and making things with code.

(4) OpenFrameWorks | Open source C++ toolkit
OpenFrameworks is an open source C++ toolkit designed to assist the creative process by providing a simple and intuitive framework for experimentation. 

(5) Unity | Game engine

• Scripts can be used to create interactions, graphical effects, and more. The official coding language is C#. 
• Create shaders by writing code.
• There are also plugins available for GPT in the Unity project in Unity Assets Store.

Quick tips

• First and foremost, we need to have a basic understanding of the software and the programming language before we ask ChatGPT to generate code. 
• Generate code chunk by chunk, or even step by step. There are two reasons to do so. 
  • First, doing so is helpful when fixing visual details or bugs. 
  • Second, considering the maximum counts of word/token for the answers we can get from ChatGPT, we want to keep an eye on the token limits as we proceed. We don't want to run out of the quota all at once. 
     

Xinyi Zhu
Motion Graphic Designer/Animator
xz3366@nyu.edu