What do you think of DeepSeek's use of distillation technology?

While the idea of distillation technology being used to enhance DeepSeek's capabilities is interesting, it’s important to clarify that DeepSeek's efficiency and performance are primarily driven by its unique architecture and optimization strategies, rather than simply extracting knowledge from other large models. Distillation is indeed a technique used in AI to transfer knowledge from larger models to smaller ones, but DeepSeek's strength lies in its innovative design, which focuses on balancing power, cost-effectiveness, and scalability. This allows it to deliver high performance without relying heavily on external models. Ultimately, DeepSeek's success is a result of cutting-edge research and engineering, not just distillation.

The greatest value of DeepSeek is that it can achieve top-level inference results with a small computing power, and as for data, it is not the focus of the problem

I think the use of distillation technology is a smart move. By extracting the essence of larger models, DeepSeek manages to retain the power of these models while significantly reducing cost and resource consumption