AR App Maps Subway Public Art for Blind and Low-Vision Riders

NEW YORK — For thousands of blind and low-vision New Yorkers who ride the subway from Jackson Heights to Coney Island, the city’s celebrated mosaic tiles and sculptures have long been a mostly visual amenity; a new augmented reality app now aims to change that by translating underground art into audio and haptic experiences, mapping the Metropolitan Transportation Authority’s permanent works so they can be found, felt and understood without sight.

The app, called Beneath, is the result of an 18-month collaboration between Open Transit Lab, a nonprofit technology research group based in Long Island City, and AccessArts Coalition, a Manhattan disability-advocacy organization, with a pilot supported informally by MTA staff in a half-dozen stations across Brooklyn, Queens and Manhattan. Using phone cameras, optional wearable haptic bands and low-frequency beacons installed near platforms, Beneath overlays spoken descriptions and tactile vibration patterns on site-specific artworks such as the mosaic panels at Astor Place and the bronze figures at Atlantic Avenue–Barclays Center.

“Public art underground belongs to everyone,” said Maria Chen, executive director of AccessArts Coalition. “When a rider with low vision approaches the Astor Place mosaics and hears the artist’s intent and feels a distinct vibration for the mosaic border, they regain an ordinary cultural encounter that had been invisible. That dignity matters in the same way a ramp matters on the sidewalk.”

Technically, Beneath combines several techniques to anchor audio and haptic cues to works that are often located in moving, noisy environments. Engineers used short-range Bluetooth beacons and computer vision models trained on smartphone camera feeds to identify artworks, then combined LiDAR scans performed by volunteers with historical catalog records to create 3-D placement maps. Haptic patterns are tied to architectural features—an ascending pulse to indicate a sculpture’s height, a continuous buzz along a mosaic border—while geofenced audio gives concise narration and optional extended commentary upon request.

In concrete terms, the pilot mapped 220 permanent artworks across 68 stations, recruited 150 blind or low-vision participants for beta testing, and ran for 12 weeks in nine neighborhoods including Midtown, DUMBO, the Lower East Side and Jackson Heights. The project was seeded with $1.2 million in philanthropic grants and in-kind support; the Open Transit Lab reports 92 percent accuracy in object recognition during daytime trials and average audio latency of 300 milliseconds, while overall user satisfaction rated 4.3 out of 5 in post-trip surveys.

“We designed the system to rely on redundancy—visual recognition, beacon triangulation and inertial data—so a missed camera read doesn’t mean a missed artwork,” said Luis Marrero, director of Open Transit Lab. “But technology can only get you so far. The most important part was co-designing descriptions with local riders, artists and station agents so the narration respects context and doesn’t reduce complex work to a single sentence.”

Riders who participated in the beta described moments that felt small and transformative. “When the app told me the name of the sculptor at Borough Hall and then guided my hand to the bronze plaque with a low hum, I cried,” said Janelle Ortiz, a community advocate from Sunset Park who is legally blind. “I ride through Borough Hall twice a week for court dates and never knew that story. It made the subway feel more like a city again rather than just a commute.”

MTA officials, who declined to provide a formal endorsement because the agency has not yet adopted a system-wide policy, acknowledged internal interest. A senior MTA planner who requested anonymity said the agency has been tracking Beneath’s technical reports and user feedback, and is exploring ways to incorporate the app’s metadata into the MTA’s existing digital asset register to streamline maintenance and signage updates without disrupting station operations.

Advocates and technologists warned that the work is far from finished. Coverage gaps remain in outlying stations, battery drain on older phones limits continuous use, and there are language and cultural translation needs for the boroughs’ multilingual communities. Privacy advocates also flagged the potential for camera-based recognition to collect incidental images of bystanders, a concern developers say they mitigate with on-device processing and automatic blurring, but which some rider groups say needs clearer rules before a public rollout.

Project leaders plan to expand Beneath to 150 stations over the next two years, add Spanish, Mandarin and Haitian Creole narration tracks, and test bus-shelter and Staten Island Ferry artworks as part of a second-phase grant application to the city. “This is a pilot, not a product,” Chen said. “If the city wants truly accessible public art, the next step is municipal investment that integrates these tools into transit planning from the start.” The team hopes that by the end of 2027, tactile and audio art guides will be as commonplace on underground platforms as the turnstiles themselves.