CEO of Stability AI Emad on X, Grok Requirements

He calls it a chunky beast.

320GB of Vram for 4 BIT model
DGX A100 = 8 x A100 for the 8 bit version.
That’s a $200,000 server rack.

If you can run grok I want to be friends lol

1 Like

My 6Gb is rumored to be falling behind. Others have wondered about if 7b or 13b models will become the norm or if we move past that quickly to other methods. Let me know if you meet that server rack guy, damn.

1 Like

Hey - 200 grand seems like a lot of money - right as I got my first non-linear editing computer (Newtek Flyer) back in the nineties - a ‘big’ video production company across town invited me to see their new five hundred thousand dollar video switcher for their linear edit suite - the production power it had did not even compare with my 10,000 dollar desktop PC - I imagine we are very shortly are going to see similar or even more radical scales of change -

To put it another way - today a plumber spends 200 thousand on a couple pickup trucks - Tesla will be putting server racks in the back of a cybertruck and giving you the truck for free - (maybe not - but you get the idea) -

3 Likes

Wow even 2 bit quant tech won’t be able to help out with that. But eventough I can not run it right now I decided to download it stubbornly anyways and we’ll see when I’ll be able to run it. Might try and run it on runpod or something similair

1 Like

Write me up to the magic internet friends list.

I’m scratching my head if someone would release it as a cloud service sooner or later - as if, anyone willing to take the risk.

2 Likes

What do you think, can it be done from “classic” pieces, like 26 x GeForce RTX 30XX, with extenders on a ASUS WS C621E Sage Extreme with 10TB SSD and some kick-ass processor?

1 Like

I’m curious if someone could ask X (Elon) to partition some cloud for Grok “researchers”, lol?

I’m also curious if collectively Grok users could make a SETI type system, encrypted, to make a reasonable compute cloud?

Open source is awesome. But we need to able to use it. Or the poor kids are just gonna watch Santa through the rich kids window. From the outside.
And that sucks…

I suppose the adventurous could sell their belongings, to pay the walls to build the code(s) for their ideas. Implement and get paid. Via a foundation they’ve set up. To afford the servers and services to pay less for the compute of what’s coming to build and implement.

Wild Times!