TLDR Tech / Data
How NetEase Games cut LLM cold starts from 42 minutes to 30 seconds
Serverless GPU autoscaling only works when model data access is prewarmed, shared, and namespace-aware. In benchmarking, startup time improved from 42 minutes to 14 minutes with Alluxio caching, then to under one minute with fully tuned Fluid prefetching.
Read the full story