They invented a hybrid attention design that drastically reduces the amount of memory needed for the KV cache at inference time. Like, dividing it by 10. And memory is a large part of the cost of inference.

Thanks for the correction, I was indeed wrong about that. I updated my comment accordingly.

Thanks for the correction! I updated my comment to mention it.

I mean, the poll was like as not a publicity stunt, to draw attention to the fact DDG is not doing AI. All the same, the fact they are making “no AI” a selling point is noteworthy.

EDIT: I stand corrected – apparently DDG does do AI presently. Hopefully they’re serious about reconsidering that, then.