How I Could Have Won My First Kaggle Competition at Google Cloud Next 2024 in Las Vegas

Apr 12, 2024

Google Cloud Next Conferences are always amazing. They are at the level of Google I/O, but on top of that, there are also a lot of after-parties and sometimes even morning events. Back on February 21st, I participated in a Gen AI Lab and Duet AI Roadshow at the Google Redwood City campus.

This was my first time at that campus and it was one of the most beautiful ones. (My only complaint could have been that there were no Level 2 chargers for visitors.)

During that event, I won a knowledge quiz game and was awarded a golden ticket to Cloud Next. Earlier in January, it was announced that Cloud Next would be held in Las Vegas for the first time in its history (normally it was held either at the San Francisco Moscone Center or a larger Bay Area Google campus event center, such as MP6). Once I learned that, I immediately reserved a hotel room and a flight, even though I didn’t have a ticket for the conference itself. Allegiant fortunately had a direct flight from Fresno, and the OYO Hotel had $11 rooms, although with all the extra fees it ballooned to about $60 a night, but that was still much cheaper than any Bay Area accommodation. The airfare bumped up the total cost, but ~$200 + ~$250 is still decent if I could snatch a conference ticket.

So I was super ecstatic when I got that golden ticket. However, after my excitement cooled down, I realized that the GDSC (Google Developer Student Club) Fresno students I was working with needed that ticket much more than I did. Last year I carpooled with Zheng Wei Ng and Ren Hao Wong to Google Cloud Next 2023 at Moscone — and I could tell a lot of stories — but both of them were about to graduate that summer, and Catherine Laserna (Google Generation and Smittcamp Scholar) would take over the baton for GDSC Fresno. I had only one golden ticket, and after a conversation with the students, we concluded that Catherine would receive the ticket.

Fortunately, later I was able to grab a 100% coupon code (after finding a $600 and then a $400 total cost coupon — which was a huge discount compared to the $2,000 full price) and also an entry as a GDG Fresno organizer for myself. So I was able to bring a four-person Fresno delegation to the conference in various ways.

At the conference, I learned about two hackathons:

  1. A Kaggle competition
  2. An AI hackathon

I focused the majority of my time and energy on these competitions. I like a challenge and it was long overdue for me to participate in a Kaggle competition.

  • To prevent any cheating, the competition dataset was kept secret, and we only received 3 sample inputs and outputs.
  • Teams needed to submit large language models that perform as well as possible on trivia questions, riddles, and poem writing.
  • The Kaggle competition environment was offline (without any internet connection), I guess to prevent off-loading of the competition dataset. Therefore, competitors needed to package everything up very tightly, because any unexpected connection attempt by any part of the submission would fail.

Looking back, this was very tricky for a first competition. Not only did I need to learn how to make Kaggle submissions, but I had to do so in a self-contained environment without an internet connection. However, on kickoff afternoon, I made my first default submission to at least take a baseline spot on the leaderboard that I could improve on later. I had the following ideas to increase my score:

  1. Try using a larger LLM. The default submission was with Gemma 1.1B.
  2. I quickly learned that the submission evaluation environment had limitations, and there was also an execution time limit. I was able to get a Gemma 7B working (FP16 INT4 GPU-optimized), but the CPU-optimized version timed out. Still, the GPU model moved me up a little on the leaderboard.
  3. Then I was thinking about a 13B Llama 2 or Mistral. I was not able to get the instance to fit a 13B model and finish in time.
  4. I spent too much time trying to tackle the model size issue. It was clear I didn’t have time for anything like fine-tuning or RAG with quiz and riddle datasets.
  5. So I focused on prompt engineering. First, I “flattered the model”. LLMs are capable of various roles, ranging from singing a lullaby to a child, speaking like a pirate, to writing scientific publications. Because they are so versatile, it is very important to tell them what their role is, because they will perform better. This technique gives less of a boost in this modern age because newer training improves the models’ default behavior so that the gap is not that big. But I still “flattered the model” by telling it what areas it is an expert in.
  6. I also used a CoT (Chain of Thought) prompt. The classic one is the “think step by step” instruction. Normally this results in the model explicitly showing the detailed steps. This competition needed short, to-the-point answers, so in addition to enabling CoT, I needed to tell the model to think through things quietly and not spell out the intermediate steps out loud.
  7. I also performed some string post-processing on the LLM’s response. I told the model: “Try to answer with no more than five words.”
  8. With these efforts, I finished 13th out of 27.

There were a lot of teams, and many of them tried fine-tuning and RAG solutions. The most important trick was a prompt engineering one: few-shot prompting. There was a guy named Chris who made some submissions, but he wasn’t competing because he was affiliated with Kaggle. He also tried RAG and fine-tuning, but his winning solution was “simply” few-shot prompting. He didn’t even flatter the model or use Chain-of-Thought. With enough good examples, few-shot prompting implicitly explains all of this to the model. My jaw dropped! I could have won if I had not forgotten the few-shot examples!

After RAG Fusion’s context resolution prompt, this competition provided yet another hard lesson on how important few-shot prompting is. As for prompt engineering in general, I’ve heard some engineering voices downplay it as pseudoscience, but it is just as important a part of a solution as any other building block (such as RAG or fine-tuning), or in this case, even more important, because I could have won simply through prompt engineering.

This blog post is too long now so I’ll talk about the AI Hackathon in the next one.

Comments loading...