📅 ThursdAI Feb 22nd - Groq near instant LLM calls, SDXL Lightning near instant SDXL, Google gives us GEMMA open weights and refuses to draw white people, Stability announces SD3 & more AI news

Hey, this is Alex,

Ok let's start with the big news, holy crap this week was a breakthrough week for speed!

We had both Groq explode in popularity, and ByteDance release an updated SDXL model called Lightning, able to generate full blown SDXL 1024 images in 300ms.

I've been excited about seeing what real time LLM/Diffusion can bring, and with both of these news release the same week, I just had to go and test them out together:

Additionally, we had Google step into a big open weights role, and give us Gemma, 2 open weights models 2B and 7B (which is closer to 9B per Junyang) and it was great to see google committing to releasing at least some models in the open.

We also had breaking news, Emad from Stability announced SD3, which looks really great, Google to pay Reddit 200M for AI training on their data & a few more things.

TL;DR of all topics covered:

* Big CO LLMs + APIs

* Groq custom LPU inference does 400T/s Llama/Mistral generation (X, Demo)

* Google image generation is in Hot Waters and was reportedly paused (refuses to generate white people)

* Gemini 1.5 long context is very impressive to folks (Matt Shumer, Ethan Mollick)

* Open Weights LLMs

* Google releases GEMMA, open weights 2B and 7B models (Announcement, Models)

* Teknium releases Nous Hermes DPO (Announcement, HF)

* Vision & Video

* YoLo V9 - SOTA real time object detector is out (Announcement, Code)

* This weeks Buzz (What I learned in WandB this week)

* Went to SF to cohost an event with A16Z, Nous, Mistral (Thread, My Report)

* AI Art & Diffusion & 3D

* ByteDance presents SDXL-Lightning (Try here, Model)

* Stability announces Stable Diffusion 3 (Announcement)

* Tools

* Replit releases a new experimental Figma plugin for UI → Code (Announcement)

* Arc browser adds "AI pinch to understand" summarization (Announcement)

Big CO LLMs + APIs

Groq's new LPU show extreme performance for LLMs - up to 400T/s (example)

* Groq created a novel processing unit known as the Tensor Streaming Processor (TSP) which they categorize as a Linear Processor Unit (LPU). Unlike traditional GPUs that are parallel processors with hundreds of cores designed for graphics rendering, LPUs are architected to deliver deterministic performance for AI computations.

* Analogy: They know where all the cars are going when everyone wakes up for work (when they compile) and how fast they all drive (compute latency) so they can get rid of traffic lights (routers) and turn lanes (backpressure) by telling everyone when to leave the house.

* Why would we need something like this? Some folks are saying that average human reading is only 30T/s, I created an example that uses near instant Groq Mixtral + Lightning SDXL to just create images with Mixtral as my prompt manager

Open Source Weights LLMs

Google Gemma - 2B and 7B open weights models (demo)

* 4 hours after release, Llama.cpp added support, Ollama and LM Studio added support, Tri dao added Flash attention support

* Vocab size is 256K

* 8K context window

* Tokenizer similar to LLama

* Folks are... not that impressed as far as I've seen

* Trained on 6 trillion tokens

* Google also released Gemma.cpp (local CPU inference) - Announcement

Nous/Teknium re-release Nous Hermes with DPO finetune (Announcement)

* DPO RLHF is performing better than previous models

* Models are GGUF and can be found here

* DPO enables Improvements across the board

This weeks Buzz (What I learned with WandB this week)

* Alex was in SF last week

* A16Z + 20 something cohosts including Weights & Biases talked about importance of open source

* Huge Shoutout Rajko and Marco from A16Z, and tons of open source folks who joined

* Nous, Ollama, LLamaIndex, LMSys folks, Replicate, Perplexity, Mistral, Github, as well as Eric Hartford, Jon Durbin, Haotian Liu, HuggingFace, tons of other great folks from Mozilla, linux foundation and Percy from Together/Stanford

Also had a chance to checkout one of the smol dinners in SF, they go really hard, had a great time showing folks the Vision Pro, chatting about AI, seeing incredible demos and chat about meditation and spirituality all at the same time!

AI Art & Diffusion

ByteDance presents SDXL-Lightning (Try here)

* Lightning fast SDXL with 2, 4 or 8 steps

* Results much closer to original SDXL than turbo version from a few months ago

Stability announces Stable Diffusion 3 (waitlist)

Uses a Diffusion Transformer architecture (like SORA)

Impressive multi subject prompt following: "Prompt: a painting of an astronaut riding a pig wearing a tutu holding a pink umbrella, on the ground next to the pig is a robin bird wearing a top hat, in the corner are the words "stable diffusion"

Tools

* Replit announces a new Figma design→ code plugin

That’s it for today, definitely check out the full conversation with Mark Heaps from Groq on the pod, and see you next week! 🫡

ThursdAI - Recaps of the most high signal AI weekly spaces is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Full Transcript:

[00:00:00] Alex Volkov: Hey, this is Alex. This week on ThursdAI, we had an hour conversation with Grok, a new and very exciting AI inference chip that exploded in popularity all over social media after showing a 5x, yes, 5x improvement in AI inference. 500 tokens per second for Lama70B and Mistral.

[00:00:32] Alex Volkov: We also talked about Google's new OpenWeights GEMMA model, Google's image generation issues, which led them to take down the abilities of this image generation to generate people. We covered new, incredibly fast SDXL lightning, and we had breaking news for Stable Diffusion 3, which is a diffusion transformer that's coming out of Stability AI.

[00:01:03] Alex Volkov: and a bunch of other news. All that after this short intro into Weights Biases.

[00:01:10] AI teams are all asking the same question. How can we better manage our model development workflow? The path to production is increasingly complex, and it can get chaotic keeping track of thousands of experiments and models. Messy spreadsheets and ad hoc notebooks aren't going to cut it. The best AI teams need a better solution.

[00:01:33] And better tools. They need Weights Biases, the AI developer platform, to unlock their productivity and achieve production ML at scale. Replace messy spreadsheets with an automated system of record for experiments.

[00:01:52] Communicate about model evaluation. and collaboratively review results across the team. Clean up disorganized buckets of models with a unified registry. Automatically capture full model lineage, all the data and code used for training and testing. Seamlessly connect to compute to scale up training. And run large scale sweeps efficiently to optimize models.

[00:02:20] Analyze the performance of large language models. And monitor LLM usage and costs with live, customizable dashboards. Get your team on the same page to bridge the gaps from ideation to production. Use Weights Biases to build, manage, and deploy better models, faster.

[00:02:41] Alex Volkov: Wasn't this cool? This is Kari. She is a original PM on the Weights Biases team. She's been there for a long time and recently we used her voice to narrate this new video that we have up on the website. And I figured I'd put it in here because it works even without the video. And I thought it was super cool.

[00:03:01] Alex Volkov: And people ask me, what does Weights Biases do? And hopefully this answers some of those questions. Now I want to switch gears and say, basically. that the format for this week is a little different. We had the folks from Grok and Matt Schumer at the beginning of the pod, and then we kept talking about everything else, like Gemma and Gemini and everything else.

[00:03:24] Alex Volkov: So the first hour of this is going to be an interview with the Grok folks, specifically with Mark Heaps and the next hour afterwards is going to be the deep dive into topics. If you're listening to this on Apple podcast, for example, you should be able to just view chapters and skip to a chapter that you'd prefer. .

[00:03:51] Alex Volkov: I want to just do a quick…