While AI is certainly here to say, a lot of us knew there was a bubble brought to you by none other than our reckless, corporate overlords. It was inevitable. All of this money being pumped for models that still cost tens of thousands to use for a simple task? The logic doesn't add up. It's worse than cryptocurrency. But Deepseek's R1 finally popped that bubble.
To put simply, Deepseek R1 uses a multi-stage process. It begins with a "Core" base model to train a new "Zero" model with reinforcement learning. Then it uses the same method with R1, while mixing in a small sample of good answers from "Zero." However, what makes this really interesting is that this learning process is not only applied ahead-of-time, but also during runtime. So when you ask it a question, it literally thinks about what the answer could be using a process called of chain-of-thought. This is what's known as a reasoning model. But the real kicker? The entire multi-stage pipeline is open source - not just R1 itself.
Supposedly, the thinking aspect is how OpenAI's o1 model behaves. Even if you're able to cough up $200/m to find out, we still don't know what goes on under the hood because it's all a supposed trade secret. Kind of ironic coming from a company with "open" in their name. A newer o3 model was going to debut this year and be even more god awful expensive. R1 forced OpenAI to switch gears and make a free "mini" version. Sure, they can tell us o1 or o3 is good, but that doesn't mean it's true. And do you want to know what's even more bloody hilarious? After the stock markets tanked, these hypocrites had the nerve to claim Deepseek stole from them. At this point, OpenAI are just role-playing as scientists.
I already saw a YouTube thumbnail for a video that said a former OpenAI engineer was giving a warning about Deepseek, and I'm, like, "oh, please." Like, yeah, I get it. China isn't exactly sunshine and roses, but OpenAI hasn't exactly been playing fair since GPT-3. Deepseek did this as a side project. They gave it all away for free for us to examine, test and run. Sure, R1 censors a lot of touchy topics related to China or their leader, but who gives a shit when it's open source? Deepseek even admitted the censorship is regional-based. Both myself and others have discovered, it is actually pretty forgiving, including the version hosted by them. I've even tested a mid-sized 14B model on a relatively old video card, and it works quite smoothly.
I am always excited about genuine push forward like this that gives us peasants something to have fun with. Up until now, things were stagnant and ChatGPT is a joke. I, for one, welcome our new open source friends from the east.