No-Code Low-Code LLM Powered Quiz Game with FlutterFlow

After the Google Cloud Next 2024 AI Hackathon I wasn’t finished at all. First off, I developed a standalone Streamlit front-end (source code) and glue functions called from the front-end to reach Agent Builder. I did my research and had to come to a solution on my own.

But that was still just a chat agent. I felt like Agent Builder doesn’t get enough credit and attention compared to Vertex AI, Model Garden, and Generative AI Studio offerings. The majority of the Google Developer Group attendees are at the beginner level, so a no-code Agent Builder-based RAG solution fits them much better. I ended up sharing my experience with Agent Builder both at the GDG Build San Jose Dual Event - Build With AI and IWD 2024 and GDG Fresno’s Build with AI event.

I also wanted to involve GDSC Fresno students Zheng Wei Ng and Ren Hao Wong, who were about to graduate. Catherine Laserna was too busy from the get-go to participate. The vision was to create a quiz game based on the sustainability knowledge dataset using no-code technologies:

1. The back-end would be Agent Builder.
2. The front-end would be FlutterFlow with some Rive animations.

FlutterFlow on the front end would allow a web platform release as well as future Android and iOS releases. We carried over my knowledge about Agent Builder gathered from the Eco Oracle chat agent. Besides all the SDG-related documents, we also indexed generic sustainability books so the knowledge base would be broader than just the SDGs.

The Playbook was rewritten to support the designed Quiz game. The LLM was reluctant to generate trivia, so it had to be guided step-by-step, and examples were created to show the expected structured JSON response so the front end could receive a well-defined object. At a later point, we also integrated the regular chat agent as an extra; this was implemented pretty easily by pointing to the Eco Oracle playbook from the Eco Trivia playbook when the user is not asking a trivia question.

I had seen FlutterFlow demos before and had experimented with it briefly. It was created by Xoogler (ex-Googler) Flutter team members, and it is quite handy once someone learns the gist of it. When I talked with Ander Dobo at Google I/O, I was happy to tell him how much I enjoyed gaining experience with FlutterFlow.

CORS errors

The biggest roadblock by far was the CORS error when the FlutterFlow front-end tried to call the glue code Cloud Functions to reach Agent Builder. I’ve come across CORS problems before at both the ODSC West Multi-Modal hackathon and the Samsung Next Gen AI hackathon. At Next Gen AI, Kevin Moore was somehow able to work around it from the Kotlin mobile app, but with the GeMMAA project, the front-end was web-based (just like now) and I remember we could not work around certain parts.

As visible in the GitHub repository, we tried many workarounds:

• Applying workaround code in the Cloud Function to specify permissive CORS behavior, with special attention to preflight HTTP OPTIONS calls. For some reason, this didn’t seem to help.
• Rewriting the function in JavaScript to use header manipulations on the function’s side or trying the special cors package to treat the issue, unsuccessfully.
• Rewriting the function as a FlutterFlow “native” Cloud Function. That is technically a JavaScript / Node 1st generation Firebase Cloud Function, but it was not deploying.
• Rewriting the function in Firebase-style JavaScript 1st generation and 2nd generation (Firebase functions somewhat differ from the GCP / Google Cloud Platform style).
• In the end, we realized that if the GCP Cloud Function was called via the Firebase-mapped URL, the nagging CORS problem disappeared.

Internal Server Errors

We also ran into 500 internal server errors bubbling up from the Agent Builder API calls:

• Sometimes the structured responses from the agent were truncated, so we needed to apply a workaround so the whole quiz round wouldn’t flop due to the last hint being truncated.
• Sometimes the agent also exceeded token limits. A possible workaround for that is changing the session ID — assuming that the overflow is caused by a lengthy conversation history (the downside is that quizzes could be repeated with a higher probability).
• During office hours at Google I/O, I was told that the current Gemini 1.0 Pro 001 would be superseded by the 002 version, which already has bug fixes. Hopefully, Gemini 1.5 Pro will arrive soon as well, which will help.
• Later, the agent often came back with a 200 OK but asked the caller to try again instead of returning a valid response.

We realized that we might have gone overboard with 23 examples (each of which can be lengthy, 1200 characters or more), and if the back-end stuffs those into the prompt for few-shot prompting, that could cause trouble. We reduced the number of examples to 6, and that seemed to resolve the issue!

Generative AI assets

We also had a polyglot experience developing the art assets. Let’s take, for example, the wind turbine animation, which needed an image of a wind turbine facing us directly, after which we had to separate the blades from the rest. We tried Gemini, DALL-E, and Runway Gen-2, and each was good in only one aspect:

• Only Gemini and EarthNet understood that the turbine should face us.
• None of them were able to understand the image editing request to delete the blades. We ended up using Google Photos for that.
• The Gemini-generated image was square, and even the EarthNet image wasn’t wide enough, so we used Runway Gen-2 to extend the art to widescreen.
• Blade separation for the animation was done manually with GIMP.

Rive Animations

The game animations — a birthday cake for a successful quiz and a waiting animation while trivia is generated — were all done with Rive. Rive has a web UI, and we looked for existing animations from the open gallery and slightly customized them. The resulting Rive animations were then integrated into FlutterFlow in a drag-and-drop manner.

Three birds with one stone

We submitted the work to three hackathons:

1. The Climate Change-Makers Challenge: 2024 (submission)
2. Google Cloud Vertex AI Agent Builder Hackathon (submission)
3. GDC (Google Developer Community) GenAI Hackathon

We didn’t place at the Climate Change-Makers Challenge, but it was very worthwhile to learn about, for example, the EarthNet portal. The project was submitted there as well. EarthNet has its own Generative AI back-end, and we ended up using their imagery for the GDC hackathon.

We also integrated the regular chat into the game on a separate page during a night hack with Ren Hao. First, we tried FlutterFlow’s Generative AI helper, but it ended up generating a page that enumerated machine-user exchanges without using a ListView. That was semantically incorrect, although at first glance it looked fine layout-wise. Then Ren Hao researched FlutterFlow’s readily available widgets and integrated one properly.

What we learned

Agent Builder agents together with FlutterFlow allow extremely rapid prototype development (assuming one works around the CORS and other API issues). If we were to repeat the development, we could be done in a day.

Hackathon Win

We ended up winning 2nd place at the GDC (Google Developer Community) GenAI Hackathon: