#inference
#inference 24 items · All topics →
NVIDIA Blog · 00:00 New Score 100 In its MLPerf debut the Vera Rubin platform led across training, post-training and agentic inference, with NVIDIA sweeping every category.
Why this: Official source · Covers inference and Agent · Published within 24 hours
arXiv cs.AI · 13:00 Score 80 S1-Evo shows that typed System-1 decision models deployed for agent routing, moderation, and authorization can still be bypassed by carefully crafted evidence-competition attacks.
Why this: Covers model and LLM · Relevant to this topic
arXiv cs.AI · 13:00 Score 75 AscendGYM sets up a benchmark for kernel optimization on Ascend NPUs that reflects real LLM training and inference workloads while factoring in cost.
Why this: Covers model and LLM · Relevant to this topic
arXiv cs.AI · 13:00 Score 75 This study compares locally deployed LLMs on scientific paper search, measuring retrieval success and latency on Mac Studio and NVIDIA A100.
Why this: Covers model and LLM · Relevant to this topic
arXiv cs.AI · 13:00 Score 70 This work stabilizes first-token and inter-token latency for interactive LLM services through tail-calibrated scheduling.
Why this: Covers LLM and inference · Relevant to this topic
arXiv cs.AI · 13:00 Score 70 This local AI server runs large language models on local devices, keeping data private, removing per-token cost and working offline.
Why this: Covers model and LLM · Relevant to this topic
arXiv cs.AI · 13:00 Score 70 This work combines mechanical inference with agents to automatically fill in the high-level specifications formal verification needs, lowering the cost of writing specifications by hand.
Why this: Covers inference and Agent · Relevant to this topic
arXiv cs.AI · 13:00 Score 65 This study applies microscaling formats to quantize recommendation and ranking models, boosting inference throughput for high-concurrency settings like advertising while preserving accuracy.
Why this: Covers model and inference · Relevant to this topic
arXiv cs.AI · 13:00 Score 65 This paper argues the Transformer revolution is not merely quantitative growth in deep learning but the emergence of a new principle of inference.
Why this: Covers inference · Relevant to this topic
arXiv cs.AI · 13:00 Score 65 This study recovers typed text from wrist surface EMG signals and shows muscle, language and typing habits all affect decoding accuracy.
Why this: Covers inference · Relevant to this topic
arXiv cs.AI · 13:00 Score 65 This work lets crystal generative models adapt at inference time, turning a strong prior into more promising candidate structures.
Why this: Covers model and inference · Relevant to this topic
arXiv cs.AI · 13:00 Score 65 This work argues text-to-image reward models need genuine visual reasoning to select better images at inference time and guide RL post-training.
Why this: Covers model and inference · Relevant to this topic
arXiv cs.AI · 13:00 Score 65 This work infers spatially resolved gene expression profiles directly from histology images, covering more genes with faster inference.
Why this: Covers inference · Relevant to this topic
arXiv cs.AI · 13:00 Score 65 This work uses pinned-key inference to make feed-forward 3D reconstruction from long videos faster and streamable.
Why this: Covers inference · Relevant to this topic
arXiv cs.AI · 13:00 Score 65 PRODiGI is a pretrained model that infers the program generating the data directly from a finite set of samples, skipping dataset-specific model fitting.
Why this: Covers model and inference · Relevant to this topic
AWS Machine Learning Blog · 05:53 Score 75 OpenFold3-preview can now be deployed from SageMaker JumpStart inside your own AWS account, forecasting the 3D structures of protein, DNA, RNA, and ligand complexes, with async inference and multi-GPU support.
Why this: Official source · Covers inference and deploy
AWS Machine Learning Blog · 03:35 Score 65 AssemblyAI runs speech-to-text inference at scale on ECS, EC2 Spot GPU instances, and an SQS-based retry architecture, cutting costs by 70%.
Why this: Official source · Covers inference
AWS Machine Learning Blog · 02:44 Score 70 Built into Bedrock, zero-instrumentation cost attribution traces each inference call back to the IAM principal behind it.
Why this: Official source · Covers inference and Tools
AWS Machine Learning Blog · 02:01 Score 65 HyperPod Data Capture logs live inference inputs and outputs to S3, feeding real traffic back into steady model improvement.
Why this: Official source · Covers model and inference
AWS Machine Learning Blog · 04:51 Score 70 Your bill can keep climbing even after prompts and app logic are tuned, because four per-request settings — max tokens, reasoning effort, service tier, and cache duration — all move the cost.
Why this: Official source · Covers inference and prompt
The guide shows how to gauge LLM inference performance with key metrics and formulas, spanning latency, token throughput, and system load.
Why this: Covers LLM and inference · Relevant to this topic
Machine Learning Mastery · 19:00 Score 60 The article shows how to pick the right LLM inference optimization technique for your workload's real bottleneck, working through a decision tree.
Why this: Covers LLM and inference · Relevant to this topic
Machine Learning Mastery · 19:00 Score 55 The article unpacks how speculative decoding works, why it delivers genuine speedups at the hardware level, and what today's implementations look like.
Why this: Covers LLM and inference · Relevant to this topic
Hugging Face Blog · 01:35 Score 65 Dharma-AI's research shows that specializing the vocabulary of small language models can cut inference cost by 77% in real applications.
Why this: Official source · Covers inference