Teaching Gemma to paint with editable code

Most image models return a finished bitmap. A person can request another image, but cannot inspect the strokes, change one compositional decision, or reuse a small part of the construction. Google DeepMind and Iacon explored a different medium with a nine-billion-parameter Gemma research checkpoint: the model would make the image by writing the program that draws it.

The project trained Gemma to produce complete JavaScript sketches using p5.brush, a painting-oriented graphics library. A sandboxed browser executed each sketch and rendered a PNG. The training system scored the rendered image, then used that judgment as a reward for the model that wrote the code.

The artifact remained editable. A user could change the palette, move an element, alter a brush operation, or continue from the generated program without asking the model to recreate the whole image. That property made the research useful beyond its visual result. It tested whether reinforcement learning could teach a language model a creative practice whose output was both visible and programmable.

Study boundary	Public description
Relationship	Joint research with Google DeepMind
Model	A 9B Gemma research checkpoint
Output	Complete `p5.brush` JavaScript programs
Training	GRPO with deterministic verifiers and visual preference reward
Optimus built	Rendering, reward, reference, prompt-optimization, and experiment pipelines
Result	Faster learning, stronger judged images, and programs shortened by more than 80%

Defining painting as an executable task

Each training episode began with a visual instruction, such as a watercolor flower with a specified subject and palette. Gemma had to return a complete program rather than a fragment or natural-language plan. The renderer ran the program in a constrained Puppeteer environment and captured the resulting image.

That environment made several properties mechanically testable. The program had to compile. It had to use the required brush library instead of bypassing the task with native drawing calls. It had to stay within an allowed interface and produce a visible artifact. A failed program could not receive credit for a persuasive explanation of what it intended to draw.

The model did not operate Microsoft Paint or control a desktop interface. It learned a code-based painting medium. This narrower boundary produced cleaner training signals and left every output reproducible as source code plus a renderer revision.

The reinforcement-learning loop

Optimus built and operated the campaign as a repeated sequence:

sample a visual prompt from the task distribution;
ask Gemma to write a complete p5.brush sketch;
execute the sketch inside the sandbox;
verify compilation and required library use;
score the rendered PNG for prompt adherence and visual quality; and
feed the combined reward into a GRPO policy update.

The project used reinforcement learning with verifiable rewards. Its reward stack mixed two kinds of evidence. Deterministic verifiers handled facts the system could check exactly, including whether the program executed and used the required API. Visual preference could not be reduced to a binary program check, so a prompt-conditioned judge compared valid renders against curated reference work.

Calling this a hybrid reward is important. Compilation is verifiable in a strict sense. Aesthetic quality is an authored judgment. The research question was whether a carefully structured combination could guide a model toward better creative programs without collapsing into reward hacking or imitation of a few examples.

The first reward improved while the paintings stalled

The initial objective combined nine signals. It included a compilation gate, a check for p5.brush use, a code-length incentive, a learned human-preference score, prompt-adherence judgments, and separate assessments of recognizability, aesthetics, technique, and depth.

Training reward increased, but the images converged on a narrow visual pattern. The programs grew longer while many renders remained flat and repetitive. The aggregate reward suggested progress that was difficult to see in the artifacts.

Optimus diagnosed the plateau by separating the reward components and inspecting their trajectories. The visual judges were correlated at roughly 0.85 to 0.95, so the objective counted similar opinions several times. The length term contributed a large share of the early reward and then saturated, leaving little useful gradient. The visual preference component that continued to vary had too little weight to steer the policy.

This was a reward-design failure rather than evidence that the checkpoint had reached its creative limit. The objective rewarded verbosity and repeated one visual opinion through several nominally different judges. Optimus used the run record to identify both problems before spending another campaign on the same rubric.

Replacing absolute taste with pairwise comparison

The first visual rubric asked a judge to assign an absolute score. Those scores compressed into a narrow range, especially when the image quality was still weak. A low score did not tell the policy which direction would produce a better composition.

The revised objective asked a relative question. For each rollout, the judge received the generated painting and reference examples matched to the prompt. It selected the stronger image, and the rollout earned reward according to the fraction of comparisons it won. Pairwise judgment produced more separation between candidates than an abstract numeric scale.

The teams built the reference pool from more than 1,600 generated images and organized them into quality tiers through human review. The strongest tier seeded the main comparison pool. Additional examples widened color and composition coverage where the initial pool was thin.

Every reference remained an output from the same programmable medium. Gemma was not being compared with an unconstrained photograph or a painting created through tools unavailable to it. The pool represented achievable examples of the task while the human ratings supplied a stable expression of taste.

Optimus simplified the final reward to four components: a compile-and-library gate, a small length check, a learned visual-preference score, and the pairwise judge. The pairwise term carried most of the weight because it provided the clearest gradient among valid renders.

With the same base checkpoint and training data, the redesigned objective reached the previous reward plateau three times faster and continued beyond it. Generated programs became more than 80% shorter while judged visual quality improved. The model learned that a better composition did not require thousands of extra tokens.

Shortening the prompt improved API reliability

The system prompt produced a separate failure mode. Early versions included a long API reference containing hundreds of lines of documentation. Gemma wrote confident, well-structured code but invented methods that did not exist. More documentation gave the model more patterns to recombine and more ways to produce plausible invalid calls.

Optimus used GEPA to evolve the prompt against the same task evaluator. Across hundreds of candidate revisions, the process converged on a short prompt with a strict allowlist of eight supported brush methods. It removed the full API manual and examples.

The shorter prompt reduced hallucinated methods and preserved more context for the visual instruction. It also made failures easier to interpret. When a program violated the interface, the evaluator could point to a small explicit contract rather than an ambiguous interaction among pages of documentation.

The finding reaches beyond this painting task. For constrained code generation, a compact operational interface may guide a model better than a large reference manual. The model needs a reliable action boundary more than it needs every possible API detail in context.

Optimus as the research operator

Google DeepMind and Iacon set the research question, checkpoint boundary, and criteria for acceptable evidence. Human judgment created the taste anchor for the reference pool and remained part of the visual objective.

Optimus performed the campaign work around those decisions. It built the sandboxed renderer, verifier stack, reference pipeline, reward calculation, GRPO experiment loop, and prompt optimizer. It ran the campaigns, retained the code and rendered artifacts, compared reward components, diagnosed the plateau, proposed the revised objective, and assembled the evidence for review.

The work was autonomous where iteration benefited from autonomy and human where the task required taste or authority. Optimus did not decide what people should find beautiful. It converted approved examples and evaluation rules into a training system that could run, fail, and improve at machine speed.

What the study established

The study showed that a compact language model can learn a visual craft through feedback on rendered programs. Hard verifiers kept outputs executable. A curated preference signal guided composition. Prompt optimization reduced API hallucinations, and the revised reward produced shorter code alongside better judged images.

The evidence applies to the fixed p5.brush environment, prompt distribution, reference pool, and visual judge used in the campaign. It does not establish a universal measure of artistic quality, general desktop control, or operation of MS Paint. Pairwise reward can also overfit to the taste represented in its reference pool, so broader styles and independent human review remain important follow-up evaluations.

The retained system makes those next experiments possible. New reference pools, brush interfaces, model checkpoints, and reward designs can be tested against the same rendering and lineage machinery. Each visual claim remains attached to the program, image, prompt, evaluator, and policy revision that produced it.