GreenTune: Energy-Efficient Low-Rank Tuning of LLMs with ThreeE Evaluation under 4-/8-bit Quantization

As web-scale Generative AI (GenAI) services become core Internet infrastructure, the energy use and carbon emissions of LLM inference are becoming a significant sustainability concern. Yet deployment decisions for conversational endpoints remain almost entirely driven by accuracy and latency, implicitly treating energy as a free resource and offering little visibility into the energy and CO₂e cost of individual requests or into how 4-bit and 8-bit precision choices affect the trade-offs between energy, performance, and task quality. We address this gap with GreenTune, a request-centric, length-aware energy and carbon accounting protocol for web-scale GenAI services and an energy-aware quantized tuning pipeline built on top of it. GreenTune samples GPU power, subtracts an idle baseline, and integrates traces to obtain per-request energy. From these measurements it derives length-dependent Joules-per-token profiles, throughput, and tail latency and converts device-side energy into per-token and per-session CO₂e estimates under standard carbon accounting assumptions. On this accounting layer, we train low-rank adapters on 4-bit and 8-bit quantized chat backbones and evaluate the resulting configurations under shared workloads to obtain fair comparisons. To support deployment decisions, GreenTune introduces the Energy--Efficiency--Effectiveness (ThreeE) composite metric, which combines energy efficiency, inference efficiency, and task effectiveness into a transparent, tunable deployment score while still exposing the underlying axes. Instantiated on ~1.1B- and 7B-parameter chat models on consumer-grade GPUs, GreenTune shows that carefully configured 4-bit variants reduce energy consumption and CO₂e per generated token by about 60% relative to 8-bit baselines, while increasing throughput by up to 3× and reducing p95 latency by more than half at comparable task accuracy. These results turn precision choices such as 4-bit vs. 8-bit from a narrow performance tweak into a system-level lever for meeting explicit carbon budgets and provide a model-agnostic path from power traces to actionable per-request and fleet-wide CO₂e budgets for web-scale GenAI endpoints.