Intel Innovation 2023 Conference Generative AI Hackathons and Challenges

The conference took place at the San Jose Convention Center. In my opinion, that proved to be a great venue: it had various spaces for sessions, a big exhibit area, a keynote section, and spaces where people could retreat if they had work to do. I drove a plug-in hybrid and was extremely happy to see that the parking garage offered an ample number of Level 2 charging stations. Here is the most comfortable keynote setup I’ve ever experienced: a small overflow stage with bean bags:

Throughout the conference, I never had to wait in lines that were too long (even though I should note that the scale of the conference is smaller than an Oracle JavaOne or Google I/O), so the conference organization and management were great. I also noticed how enthusiastic and helpful all the Intel employees were. That also applied to interactions among themselves, not just with conference attendees. This created a great atmosphere for the whole conference, and I’d certainly consider Intel as a workplace based on what I saw.

I focused my efforts on the “Intel AI Innovation Bridge” Generative AI hackathon, as well as side challenges such as the Gen AI Rock Star challenge. There were a few categories, and I couldn’t decide on the first day which one I wanted to target. On one hand, I wanted to experiment with multimodal generative AI. Since my wife and I were trying to find a home for a cat in our apartment complex, I was thinking about an app that could boost animal shelter adoption rates with enhanced pet listings. Google researchers introduced a technique called DreamBooth, which can fine-tune a Stable Diffusion model on a specific person, animal, or subject with only a few example photos. After fine-tuning, the model can generate images starring the subject in any requested scenario. Animal shelters could enrich their photographs with cute scenarios for each pet, and I thought this could be a great opportunity to port DreamBooth to Intel Max GPUs or Intel Habana Gaudi2 accelerators.

On the other hand, I wanted to explore knowledge chat agents for the startups I work for (Sportsboard and ThruThink). That might not be as cute as a cat or dog, but it would be much more connected to my everyday work and could potentially yield immediately applicable results. For ThruThink, I could control the output format of our help database and tested HTML and Markdown versions too. I went with Markdown because I had seen semantic chunking with that format. I even contributed to an open-source question_extractor project to avoid hammering embedding APIs too heavily, and extended the supported API targets (AnyScale) and Q&A formats (PaLM 2, Azure OpenAI, …).

At the end of the first day, I decided to resist the DreamBooth cuteness and go with the high-value business LLM agent route. That left less than half a workday before the project submission deadline on the second day of the conference. So at the end of the day, I studied the example datasets we received for fine-tuning and discovered some discrepancies (for example, Questions and Answers were flipped in certain parts of the dataset). The takeaway is: just like with any AI/ML task (or more generally any software project), everything starts from the data, even for generative AI. The same data issues emerge, and often a large portion of the time is spent massaging data. At the time of the conference, cnvrg’s LLMaaS (LLM as a Service) by Intel was in the testing stage. The LLMaaS service provided a no-code environment to fine-tune and customize agent solutions with ease. Due to the early stage of this novel product, I used two toy datasets to fine-tune an LLM, and I applied RAG (Retrieval-Augmented Generation) with the help of a Pinecone vector database using OpenAI’s embeddings API. I also got familiar with Gradio and similar front-end solutions, and ended up using Gradio to productionize the agent. In the end, the judges were impressed with my work and I won the LLM category.

There was also a Gen AI Rock Star challenge. This was about an hour-long challenge where participants were tasked with generating a poster or album cover for an imaginary rock band, along with lyrics for a song by that band. I teamed up ad hoc with a fellow attendee, Ronald Randolph. We were provided with a Gaudi2-accelerated deep learning AWS EC2 instance equipped with a Runway 1.5 text-to-image Stable Diffusion model, as well as a Vicuna 7B-based LLM. One key to our win was increasing the inference steps of the Stable Diffusion model from 10 to 50, resulting in much higher quality images. The EC2 instance was inaccessible for a while; in the meantime, we experimented with other LLM models for lyrics, such as ChatGPT and PaLM, and Runway’s web platform for image generation. We used the LLM to come up with the band name (Nerdcore Annihilation) and the song title (Xeon Fury). The diffusion model couldn’t render text on the image precisely, so we used Runway’s web-accessible editing feature to add an old-school Intel Inside logo and modify the center image of the front cover. The back cover was purely generated by the Gaudi2 instance (notice how the wafer is a pastry wafer and not a silicon wafer; we left that in as a pun). We adjusted the prompt many times—prompt engineering is extremely important. I used my geek knowledge to provide desired hardware keywords for the lyrics, such as MMX, AVX, AVX2, AVX512, AMX, Xeon Phi, Gaudi 2, Optimum Habana, and the freshly announced 288-core Xeon. We mixed the best parts from various attempts to create a hilarious final set of lyrics.

The conference was a huge blast, and I hope to be back in 2024! Of course, it’s even sweeter when you have some wins under your belt. It was amazing to see what other teams accomplished as well. For example, the three gentlemen in the center of the photo are working on an AI solution that predicts whether a patient might fall in a hospital by analyzing video feeds. Fall accidents in hospitals can result in serious injuries such as hip fractures, taking an unwanted toll on already sick patients. I could tell that those fellow engineers had a very good chance at having a great run with that extremely useful application. Their real win wasn’t just the 14900K CPUs, but rather the startup-specific support they’ll receive from Intel to succeed.