No-Code Low-Code LLM Powered Quiz Game with FlutterFlow

After the Google Cloud Next 2024 AI Hackathon I wasn’t finished at all. First off I developed a stand-alone Streamlit fornt-end (source code) and glue functions to call from the front-end to reach the Agent-Builder. I did my research and had to come to a solution on my own.

But that was still just a chat agent. I felt like Agent Builder doesn’t get enough credit and attention compared to Vertex AI, Model Garden, and Generative AI Studio offerings. The majority of the Google Developer Group attendees are at the beginner level, so a no-code Agent Builder-based RAG solution fits them much better. I ended up advocating my experience with Agent Builder both at the GDG Build San Jose Dual Event - Build With AI and IWD 2024 and GDG Fresno’s Build with AI event.

I also wanted to involve GDSC Fresno students Zheng Wei Ng and Ren Hao Wong who were about to graduate. Catherine Laserna was too busy from the get-go to participate. The vision was to create a quiz game based on the sustainability knowledge dataset and use no-code technologies.

1. The back-end would be Agent Builder
2. The front end would be FlutterFlow with some Rive animations.

FlutterFlow on the front end would allow web platform release, as well as future Android, iOS releases. We carried over my knowledge about the Agent Builder I gathered with the Eco Oracle chat agent. Besides all the SDG-related documents we also indexed generic sustainability books so the knowledge base would be wider than just the SDGs.

The Playbook was rewritten to support the designed Quiz game. The LLM was reluctant to generate trivia, so it had to be step-by-step guided, and the examples were created to show the expected structured JSON response so the front end could receive a very well-defined object. At a later point, we also integrated the regular chat agent as an extra and that can be implemented pretty easily by pointing at the Eco Oracle playbook from the Eco Trivia playbook when the user is not asking about a trivia question.

I’ve seen FlutterFlow demos before and experimented with it briefly. It was created by Xoogler (ex-Googler) Flutter team members, and it is quite handy once someone learns the gist of it. When I talked with Ander Dodo at Google I/O I was happy to tell him how I enjoyed getting experience with FlutterFlow.

CORS errors

The biggest roadblock by far was the CORS error when the FlutterFlow front-end tried to call the glue code Cloud Functions to reach the AGent Builder. I’ve come across CORS problems before at both the ODSC West Multi-Modal hackathon and the Samsung Next Gen AI hackathon. At the Next Gen AI Kevin Moore somehow was able to work it around from the Kotlin mobile app, but with the GeMMAA project the front-end was web (just like now) and I remember we could not work around certain parts.

As visible in the GitHub repository, we’ve tried many workarounds:

• Applying a workaround code in the cloud function to specify permissive CORS behavior with special attention to preflight HTTP OPTIONS calls. For some reason, this didn’t seem to help.
• The function was rewritten in JavaScript to use JavaScript header manipulations on the function’s side or try special package cors to treat the issue unsuccessfully.
• The function was rewritten in FlutterFlow “native” cloud function. That is technically a JavaScript / Node 1st generation Firebase cloud function but it was not deploying.
• The function was rewritten in Firebase style JavaScript 1st generation and 2nd generation style (Firebase function somewhat differs from the GCP / Google Cloud Platform style).
• In the end there was a realization that if the GCP cloud function is called via the Firebase mapped URL then the nagging CORS problem disappeared.

Server Internal errors

We also ran into 500 internal server errors bubbling up from the Agent Builder API calls.

• Sometimes the structured responses from the agent were truncated so we needed to apply some workaround so the whole quiz round wouldn’t flop in favor of the last hint to be truncated.
• Sometimes the agent also overflows token limits. A possible workaround for that is changing the session ID - thinking that the overflow is caused by a lengthy conversation history (the downside is that there could be repeated quizzes with a higher probability).
• At an office hour at Google I/O I was told that the current Gemini 1.0 Pro 001 will be surpassed and the 002 version which already has bug fixes. Hopefully the Gemini 1.5 Pro will arrive soon as well and that will help.
• Later the agent often came back with a 200 OK but was asking the caller to try again instead of a valid response.

We realized that we might have gone overboard with 23 examples (each of which can be lengthy, 1200 characters or more), and if the back-end stuffs those into the prompt for a few-shot prompting that could cause trouble. We decreased the examples to 6 and that seemed to help the issue!

Generative AI assets

We also had a polyglot experience developing the art assets. Let’s take for example the wind turbine animation, which needed an image of a wind turbine perfectly facing us and then separating the blades from the rest. We’ve tried Gemini, Dall-E, and Runway Gen-2 and each one was good only in one aspect:

• Only Gemini and EarthNet understood that the turbine should face us
• None of them were able to understand the image editing request to delete the blades. We ended up using Google Photos for that
• The Gemini-generated image was square, and even the EarthNet wasn’t wide enough, so we used Runway Gen 2 to extend the art to widescreen
• The blade separation for the animation was done manually by GIMP

Rive Animations

The game animations of the birthday cake for a successful quiz and the waiting animation, while the trivia is generated, were all done by Rive. Rive has a web UI and we looked for existing animations from the open gallery and slightly customized them. The resulting Rive animations could be integrated into FlutterFlow in a drag-and-drop manner.

Three birds with one stone

We nominated the work into three hackathons:

1. The Climate Change-Makers Challenge: 2024 (submission)
2. Google Cloud Vertex AI Agent Builder Hackathon (submission)
3. GDC (Google Developer Community) GenAI Hackathon

We haven’t been placed at the Climate Change-Makers Challenge, but it was very lucrative to learn about for example the EarthNet portal. The project was submitted there as well. EarthNet has its own Generative AI back-end and we ended up using their imagery for the GDC hackathon.

We also integrated the regular chat into the game on a separate page during a night hack with Ren Hao. First, we tried FlutterFlow’s Generative AI helper, but it ended up generating such a page that enumerated some machine-user exchange but not in a listview. That is semantically incorrect although for a first cursory view, it looks fine layout-wise. Then Ren Hao researched FlutterFlow’s readily available widgets and integrated one.

What we learned

Agent Builder Agents together with FlutterFlow allow extremely rapid prototype development (given that someone figures out the CORS problem and other problems). If we’d repeat the development we could be done in a day.

Hackathon Win

We ended up winning 2nd place at the GDC (Google Developer Community) GenAI Hackathon: