Case study
Naetha Phigram: a realistic 3D model of a building from an iPhone
Indoor scanning with a LiDAR phone, a surface growing on the Mac during the scan and a textured model to view in the browser. A building is made of room scans joined into one model.
- Type
- Kol.ski's own product
- For whom
- Housing communities, property managers, service and survey companies
- Scope
- iOS scanning app, macOS app for processing and joining scans, textured reconstruction, GLB export and a browser preview
- Status
- Working prototype, first scan of the target building, before the staircase scan
Challenge
A housing community or its manager rarely has an up-to-date model of the building. Design drawings do not show what the staircase looks like today, and one-off scans from apps end at a single room or a single walk-through.
Let's scan the car park today and two floors up tomorrow.
A building cannot be scanned in one session. The phone loses its bearings on a long walk and on the stairs, similar floors are easy to confuse, and every new scan has to land in the right place in the model instead of breaking what is already there.
Solution
The iPhone records the image, depth and camera position and sends the scan to the Mac over a cable or over Wi-Fi on the local network, already during the scan. The MacBook fuses a surface from every frame and shows it as it grows, 11 ms after the frame is saved over the cable. After the scan it computes a model textured from the photographs.
Scanning without taking photos
The operator walks through the room and the app picks keyframes itself after every 10 cm or 10° of movement. Exposure and white balance are locked for the scan, so the photographs share the same colour.
The phone as a buffer
A minute of scanning takes 77-94 MB. A scan sent to the Mac and confirmed disappears from the phone, and after a broken connection the transfer resumes from the missing frame. From the Mac the scans go to an archive on a shared drive, checked with SHA-256 checksums.
A building from rooms
Each room is a separate scan that shares a part with the next one. In Studio the scans are laid out side by side and joined with a button: the program fits them itself, offers a candidate to check when the shared part is small, and refuses when it is too small. Scans make up projects, and projects make up the building's catalogue.
Zones to rescan
Studio points out holes, surfaces seen too little and the ends of the scan. The zones are numbered on the model in Studio and on the preview page, and a rescan joins the project as another scan.
The model in the browser
The browser export has 150,000 triangles, and the preview page carries all its libraries itself, with no outside connections. It opens on an iPhone from a QR code shown in Studio.
Points, regions and versions
A point with an identifier, a name and a description can be marked on the model's surface. The model can be cut into regions, and an approved version gets checksums and no longer changes.
Result
- 2devices in the whole process: an iPhone with LiDAR and a MacBook
- 0photos of the building sent to external services
- 99%of the corridor's surface in the model; off-the-shelf engines: Apple Object Capture 44%, COLMAP with OpenMVS 16%
- 3seconds for a textured model of a room from 447 frames
- 0.5cm from the true position after joining two parts of a room scan
- 9.7mm median depth error in the corridor with drift correction (15.4 without it)
What works
- Recording the image, depth and camera position on an iPhone 16 Pro with exposure and white balance locked.
- Sending scans over a USB cable and over Wi-Fi, with acknowledgement and resumption, also live.
- A live preview of the surface and of the finished model in Studio, with orbiting, flying and walking inside.
- Reconstruction to a textured GLB, drift correction as an option and a model quality measure on held-out frames.
- Joining scans into projects and the building's catalogue, zones to rescan, points, regions and versions.
- Browser export, a scan archive and all of Studio's work also through the
phigram studiocommand, without a window. - First scan of the target building: a corridor, 490 frames in 212 s, sent over Wi-Fi without a dropped connection.
Ahead
- Scanning the staircase: two levels with the flight of stairs between them, then the whole staircase from the car park to the top floor.
- Drift correction on by default, if it holds on a second corridor scan and in the staircase.
- Rescanning from the phone: the project open in the iOS app, with the zones to rescan over the camera image.
- Masking people and number plates in the photographs.
Naetha Phigram is not a geodetic survey: we state the model's accuracy from an error report on control points.
Architecture
Each part does one thing. Everything the Studio window does can also be done with the phigram studio command.
Phigram Capture
Records the image, depth and camera position, sends scans and deletes them once confirmed, shows the models from the Mac
- Swift
- SwiftUI
- ARKit
- RealityKit
- LiDAR
Phigram Studio
Catalogues, projects and scans, live preview, joining scans, zones, points, regions and versions, export and archive
- Swift
- SwiftUI
- Metal
Phigram Processing
Depth fusion, drift correction, mesh, textures from photographs, scan alignment, quality measure, export
- Swift
- TSDF
- ICP
- glTF 2.0
Phigram Kit
Session format with a validator and the transfer protocol shared by both apps, over a cable and over Wi-Fi
- Swift
- Network
- Bonjour
- usbmuxd
Lessons
- The phone's position is a starting point, not a result. The error grows with the length of the walk, so a long scan has to be split and the final model needs camera pose correction.
- Off-the-shelf engines get lost on plain walls. In the corridor Apple Object Capture rebuilt 44% of what the LiDAR measured, COLMAP with OpenMVS 16%, and our depth-based pipeline 99%, so the pipeline stays our own.
- A change without a measurement is a guess. Two intuitive improvements, picking sharper photographs for the texture and fitting frame positions to the model, made the model worse once measured and were switched off.
- A building is not scanned in one walk. Room scans with a shared part, joined on the Mac, are simpler for the operator and more robust to error than one long session.
- The phone is a buffer, not an archive. A scan has to reach the Mac quickly, and the source data moves on to a shared drive; the results stay on the Mac.
- Texture colour depends on the camera. Without the lock one room scan had 24 different exposure times, with the lock it had one.
- Missing data stays missing. We do not fill in walls generatively in a model meant to describe a real building.
A building to scan?
Indoor models for housing communities, property managers and companies, ready for element identification and utility layers. Write to us if you want to join the pilot.
The first conversation is free: 30 minutes online, no preparation and no commitment.