Museum Indoor VPS
ongoingA web-based indoor positioning system for museums — room scanning builds a 3D point cloud map, feature matching and PnP relocalize the camera, and a voice LLM with tool access guides visitors natively.
Notes
This project is a VPS (Visual Positioning System), web based, as local as possible, for indoor positioning and guidance in museums (or other indoor contexts).
It builds on several parts: - A user-friendly administration interface for staff to scan rooms (with assisted scanning), add entrances and exit points, add artworks or POIs with their descriptions, and connect rooms together. - A user side positioning system able to relocate the camera in the map from a smartphone picture and position POIs and exits relative to the user position and orientation. - A voice LLM assistant with localization tooling able to seamlessly understands the spacial context of the user and the position of elements around them.
The whole system is based on: - xfeat for feature extraction - lightglue for feature matching - glomap for SFM and point cloud generation - a custom PnP implementation for camera relocalization - arcore for point projection at scanning time (mainly for user scanning guidance) - cellphone sensors for movement detetction