The AI infrastructure revolution happening in your community: What states, localities, and utilities need to know

by CTC Technology & Energy

By Jacob Levin and Joanne Hovis

While headlines focus on massive hyperscale data centers in remote locations, a quieter but significant infrastructure shift is unfolding closer to home. Smaller AI computing centers are beginning to appear deep in America’s towns and cities—at existing telecommunications facilities, at the base of cell towers, and even at power substations—often without local and state officials realizing it.

These edge facilities will have nowhere near the power or water requirements of hyperscale facilities—and could be built in ways that minimize water usage, make the electric grid more resilient, and support local community development. How they are built—and how the community is impacted for better or worse—will depend on the incentives and disincentives created by state and local policy.

For state and local governments, therefore, understanding this trend and the fiber infrastructure it demands is essential to ensuring their communities are not left behind in the next wave of technological and economic transformation. For utilities with fiber assets or an interest in developing them, edge facilities present a considerable business opportunity—one in which their infrastructure could be the critical enabler. For communities generally, fully understanding the AI infrastructure picture requires understanding this new generation of edge deployments.

A second wave of AI investment is coming to your community

Most public discussion about AI infrastructure has centered on the hyperscale data centers being built to train large language models. Training—the process of feeding models huge amounts of data in order to build the neural networks that can then predict and generate useful outputs—has driven the first, massive wave of AI capital expenditure.

But infrastructure requirements are changing as AI models move from this training phase into widespread real-world use known as “inference”—where users and their AI agents interact with models to get results, process video and sensor data, generate analysis, and run intelligent applications. While the hyperscale buildout continues, this shifting use is leading to a growing share of AI investment flowing to smaller edge sites, newly built or upgraded to support AI inference workloads.

The emerging wave of investment in AI edge sites is being driven by a few related trends. Inference, particularly for latency-sensitive applications, demands computing resources located closer to the people and businesses using them. This can create traffic flows that are inefficient to transport back and forth to distant cloud locations.

As models interact with a growing number of data-generating cameras and sensors, the cost and latency associated with sending those data to centralized locations incentivizes more processing to happen closer to users. And for certain industries, data privacy, security, and regulatory requirements constrain where inference can take place.

While the first wave continues to fund mega-facilities, these trends—alongside the power constraints and community opposition limiting hyperscale data center growth — are spurring a second wave of AI infrastructure investment.

That second wave is beginning to distribute smaller AI computing resources across communities in a variety of forms:

  • Micro-data center installations on or near-premises of hospitals, warehouses, and factory floors;
  • Upgraded telecommunications facilities (central offices, at the base of towers, in huts and colocation facilities); and
  • New small data centers built at utility substations and other locations with stranded power.

It’s too early to tell where the bulk of this investment will be concentrated, but the growing revenue opportunity is leading operators to consider ways to upgrade existing edge facilities to support AI workloads. Just a few examples:

This type of investment is reaching communities across the country—and many of these sites are built to scale, meaning both power and communications needs will grow over time.

Fiber is an indispensable ingredient 

Power requirements for AI get plenty of attention, but fiber has to be there right alongside the electricity. Companies can deploy enormous amounts of computing hardware, but without high-capacity fiber, they cannot connect that computing power to users.

The relationship between fiber infrastructure and AI usability is already measurable. Research indicates that people who access AI tools over fiber connections interact with them more frequently, find them more effective, and discover more innovative applications—not because fiber users are inherently more interested in AI, but because the infrastructure they are using removes the friction that constrains users on lesser connections.

This dynamic will intensify as AI applications become more data-hungry and latency-sensitive. Increasingly, user experiences with AI applications will be shaped by how fiber and computing resources are distributed across their local area.

Consider a practical example: A city’s police body cameras and surveillance video generate large volumes of data. Sending that footage to a centralized cloud for processing can be expensive and may have high round-trip latency. By processing video data locally with an AI model hosted on-site, a city can reduce telecommunications spending and operationalize the data in real time to support faster decision-making.

The analogy to earlier technology waves is instructive. Just as communities with cable broadband infrastructure had access in the early 2000s to internet capabilities that communities without cable did not, the places with sufficient fiber density and proximate AI computing infrastructure will have access to AI-powered tools and economic opportunities that other communities will not.

Localities and utilities should prepare now for new opportunities

The emerging landscape of distributed AI infrastructure creates an opportunity for state and local policymakers and a business opportunity for utilities. Here are several considerations that should be on their radar.

For localities

  • Understand how to influence what is already happening in your community. Edge AI infrastructure is appearing in telecommunications facilities, commercial buildings, and other existing sites, sometimes without generating the kind of public attention that a hyperscale data center proposal would. Policymakers should consider updating zoning ordinances and permitting processes to incentivize deployment patterns that maximize the benefits to the surrounding community. Informed regulatory frameworks can encourage beneficial development while protecting community interests.
  • Prioritize fiber density as an economic development strategy. Communities that want to ensure their local businesses and institutions can leverage AI applications should treat metro fiber infrastructure as foundational. Encouraging the buildout of robust local fiber networks—and ensuring that edge computing facilities can be served by those networks—should be a strategic priority.
  • Encourage edge infrastructure providers to be responsible community partners. The goal is not to approve infrastructure proposals uncritically, but to ensure that AI infrastructure deployments happen in ways that are most beneficial for the wider community—including being responsible grid citizens and contributing to local economic vitality.

For utilities

  • Understand how to leverage your existing or potential communications assets. Edge AI represents a potential business opportunity for utilities to leverage fiber, conduit, and infrastructure paths. Commercialization of existing assets could create revenue streams to fund new communications assets to meet the utility’s needs.
  • Develop strategy for the alignment of your power and communications assets. Edge AI will require robust power and robust fiber-based communications, but in more manageable quantities than required by hyperscale data centers. For utilities, this can mean twin revenue streams distributed across an electric service territory rather than the more challenging demand of connecting a large data center.
  • Incorporate edge AI investment into grid planning. As AI investment flows into edge sites, forward-thinking utilities will find ways to steer a portion of these investments to help upgrade power and communication assets, creating a more resilient and efficient grid while reducing the cost burden on ratepayers.

Are you ready?

The AI infrastructure story is no longer just about hyperscale data centers in distant locations. A distributed network of smaller edge facilities is emerging in many communities—and fiber connectivity is an essential enabler that determines whether a community can participate in this transformation.

States, localities, and utilities that understand these dynamics will be better positioned than those that wait. Communities and utilities should prepare now for the evolving AI infrastructure landscape.

This article was originally published by the Benton Institute for Broadband & Society