2024 has been a year of rapid development and iteration at Auki, showcasing what sets us apart in a space often dominated by hype: a relentless focus on building real technology.
As the end of the year approaches, we’ve taken stock of what we’ve done over the past 12 months. It’s mind-boggling to see the tangible achievements our developers have delivered this year.
On the software side, things that started as ideas came to fruition and developed into sophisticated solutions ready for market.
We’ve launched a token, announced partnerships and pilots with retail customers.
Here is a recap of our year. At first we hadn’t intended to organize it by month, but when we sat down to summarize everything, we realized how much we’d forgotten and how much we had for each month. It’s a testament to the breakneck pace of development and to how hard the team has worked.
January
Level 10 Research Center: We got access to the entire 10th floor of CTF Life Tower in Hong Kong thanks to our backer Baboon VC, and set up a demo space with grocery shelves both for faster R&D and for showcasing the power of collaborative spatial computing.
Domain topography & occlusion: Before implementing occlusion volumes (which correspond to physical features like shelves), all digital assets in a domain were visible in app, leading to a cluttered and unnatural view. With domain occlusion, digital information can be correctly hidden behind physical objects. This leads to a more natural and immersive experience, and saves battery compared to running LiDAR-based occlusion on device (which only has a ~5 meter range anyway).
Domain visualization: This first iteration was capable of showing navmeshes, or walkable areas, and products in a web tool for Cactus.
AI-generated tasks in space: By running empty shelf detection with computer vision and raycasting against domain topography (i.e. occlusion volumes), we created the first automated tasks in domains that staff could navigate to. This was the first milestone in giving AI spatial awareness.
Cross-domain navigation in Gotu: We enabled navigation from one domain to a neighboring one. This effectively linked them together into an interconnected network, showcasing the viability of the Auki Network and Gotu for large scale navigation across a network of domains.
Decentralization: We open sourced our relay server (previously called Hagall), marking a major step forward in decentralization and transparency.
March
Rightful Ruler: This was the idea at the very beginning of everything Auki — a shared AR measurement tool for tabletop games such as Warhammer 40k and Age of Sigmar. We finally launched the app on iOS and Android, fulfilling a four year long promise to Kickstarter backers.
Whitepaper: After years of R&D, it was time to officially retire our old blackpaper and publish a new whitepaper. This is not your typical whitepaper! Highly recommended reading.
Domain reconstruction: Our goal with reconstruction was to take sensor data when capturing a domain and turn it into a rich 3D representation of the space. Our first approach to 3D scene reconstruction was based on extracting volumetric features from a voxel-based occupancy grid. After scanning the domain, we prototyped automated occlusion volume generation from the volumetric features.
Gaussian splats: This is a different approach to rendering domains in 3D. Using Gaussian splats, we created a video preview of a navigation path through a domain. The result creates a more photorealistic representation of a domain particularly in motion.
April
Product maps: We created our first product maps by scanning barcodes and raycasting against occlusion volumes to get their poses. This was the first milestone in solving the retail industry’s big problem of not knowing where products are in their stores. It’s also the foundation for in-store product navigating.
Blind navigation: By adding audio instructions to our Gotu navigation app, we created a demo of navigation for the blind. This, particularly when combined with our cross-domain navigation demo, highlights the potential accessibility benefits of spatial computing. It doesn’t just give robots, devices and AI access to physical space, but enables them to assist individuals with accessibility needs in that space as well.
Collaborative SLAM: Using RGB (video) and IMU data from an iPhone, we remotely analyzed the path taken by the iPhone in our first demo of remote SLAM (simultaneous localization and mapping), and achieved an accuracy within 2.5%. This marked the beginning of being able to offload computationally heavy spatial reasoning to an edge device, thereby saving battery life on the iPhone. These battery savings will later apply to robots as well, which we expect will significantly extend their range.
Token2049: Our web3 team attended Token2049 in Dubai and made some new friends in the AI and DePIN spaces that have since turned into fruitful relationships.
May
Domains SDK release: Along with the public launch of our Domain Management Tool (DMT) app for setting up domains, we released major updates to our SDKallowing other developers to build on domains for the first time.
React apps: Our apps got a huge UI/UX upgrade when we finally completed the move away from Unity-based app UI to React Native. With these upgrades only the AR features remained in Unity, while the remaining UI became much slicker and much more enjoyable to use.
Web3 integration: We integrated web3 wallet functionality into DMT so users could connect their wallets and burn (testnet) tokens for network credits. This was a major milestone in implementing the burn-credit-mint mechanisms that underpin our token economy.
RACE * Smart Spaces: Visitors at the Retail Asia Conference & Expo loved the empty shelf detection solution we showcased and headed to our demo space after the event for a panel on smart spaces which we hosted together with the Swedish Chamber of Commerce.
June
Pinger app: We launched a Telegram mini app called Pinger to collect latency data for our Relay network. This is important because spatial computing requires ultra-low latency to handle real time interactions, and we’re making the bet that DePIN can beat centralized cloud giants in providing the lowest latencies to users around the world. In its first week after being soft launched, over 11,000 users registered on the app and logged over 14 million latency reports.
Server-side pose refinement: Drift occurs in AR when tracking of the device’s position or orientation gradually becomes more inaccurate over time, leading to misalignment errors when setting up large domains. To combat this, we developed a server-side pose refinement algorithm that corrects drift. Below was our first of many breakthroughs; the yellow and pink lines correspond to the original path from ARKit and the refined path, respectively.
Cactus launch: We combined several different retail operation apps (for spatial tasks, product layout, and product search) into one rebranded app called Cactus and started piloting it.
Multi-store pilot: The team traveled to Sweden to set up domains in 5 stores for our first multi-store Cactus pilot.
July
Domain setup & reconstruction: We made huge progress toward making the domain setup process easier and more accurate. Instead of individually scanning hundreds of QR codes and manually correcting drift in their poses, we developed a new approach:
Multiple users walk around filming the domain with overlapping QR codes between different recording datasets.
Each dataset gets put through the server-side pose refinement algorithm.
All of the refined datasets get stitched together.
It then goes through a global refinement process.
We extract a sparse point cloud of the whole store.
Portal kits & McKenna: In another big step, we launched portal kits (packs of QR codes for domain set up) and our AR digital art app McKenna so that anyone can set up domains in their own spaces and decorate them with NFTs, images/videos, 3D assets, and text notes. McKenna, like Cactus, shows the potential of persistant AR, spatial computing and the Auki network for real world applications.
Barcode scanning: Instead of individually scanning barcodes with an external reader, we began detecting price tags in posed images from store scans and running them through a barcode reader SDK. This significantly reduced the amount of work involved in creating product maps. This has the potential to save tens of hours for an individual store and multiples of that for retail chains.
August
Whereable part 1: We prototyped a lightweight wearable device that sits around the neck. In a breakthrough demo, we showed it entering a domain, sending its RGB and IMU data to a Motion server running on an edge device which computed its pose. The device simultaneously sent images to a Vision server running empty shelf detection. When an empty shelf was detected, the Cactus backend performed a raycast against domain topography and created a task there for staff to restock the shelf. This was the first demo of so many different devices and servers collaborating on spatial AI in a domain.
Whereable part 2: We paired Cactus and the posed wearable device with a pair of smart glasses that could display tasks as well as navigation arrows, so that store employees can navigate hands-free to a task and know what to do when they arrive.
Product heat maps: This was our first shot at visualizing product sales data in the form of heat maps, and the beginning of our foray into shelf data analytics which we believe is a billion dollar opportunity. The heat map displays product sales color-coded based on sales volume within a set period of time. (Note also how far we’d come on domain visualization compared to the first image from January!)
Automated facing detection: In another big development for Cactus, we demoed automated planogram compliance checking by counting facings, or how many units of a product are visible on a shelf. We achieved this without object recognition, which is difficult given many different factors including how many different items a typical grocery store stocks. Since planograms dictate the number of facings each product should have, counting facings makes verifying this aspect of planogram compliance an automated process.
Domain setup time: By using the new domain setup process of filming the store, we reduced the setup time for a 4,000 sqm store from 2 working days to less than 4 hours for a team of 3 people. This marked a huge improvement in scalability.
libp2p: Making machines publicly accessible with domain names, SSL certificates, static IP addresses and port forwarding has been one of the biggest struggles for our node operators. To address this issue, we made good progress with libp2p for P2P communication, with the goal of using the Relay network to help expose otherwise inaccessible machines to the network.
MEXC: We listed $AUKI on our first centralized exchange, MEXC.
Token2049 & KBW: Following our successful trip to Dubai in April, we expanded our presence at web3 events by also attending Token2049 Singapore and Korea Blockchain Week.
Map saturation: After collecting over 200,000,000 latency reports with the Pinger app, we switched from a cloud saturation reward structure (in which only nodes hosted on the top 4 cloud providers were rewarded) to a map saturation reward structure (in which all nodes are rewarded based on location) in order to encourage better geographic spread. Our hope is that by decentralizing the network more, we’ll be able to demonstrate that DePIN achieves better latency results than centralized cloud providers.
October
Domain servers: We publicly launched domain servers, allowing the community to host each other’s domains and of course also allowing anyone to host their own domain data for maximum privacy.
Business development: Nils and the BD team successfully negotiated our first 7 digit ARR deal with one of Sweden’s largest retailers (the same one we set up a 5-store pilot with). Deal terms were agreed upon and now it’s just a matter of getting through the lengthy legal process.
Automated occlusion volumes: Though we tried this previously with our volumetric feature scans, this was the first time we extracted automated occlusion volumes from a point cloud reconstruction, easily shaving off at least an hour per store setup.
AWE Vienna: The HK government was the largest sponsor of Europe’s largest XR event, and they flew the Auki team out there to be part of their pavilion.
Robotics: We decided to focus heavily on robotics in the coming year in order to showcase the competitive advantage of giving robots a shared, external sense of space and making them interoperable. To achieve this, we ordered $85k worth of cutting edge robots which we expect to be delivered in Q1 of 2025.
Cactus AI: In this demo, we integrated an LLM into the Cactus backend so that we could chat with it about product sales and revenue data. Again, see the screenshot to get a sense of how much store/shelf visualization has improved since both September and January.
Ambassador program: Immediately after onboarding our first 20 ambassadors, they leveled up our marketing game tremendously!
Floorcraft: This multiplayer AR game was one of the first apps we created two years ago to showcase our SDK’s shared augmented reality features. Since we later decided to focus on building the protocol itself and going to market with Cactus, we decided to open source Floorcraft to give someone else the opportunity to build on its potential.
December
Multi-level domains: Whereas before individual domains could only be on one floor, we found that to be too limiting and upgraded domains to support multiple floors. This replaced our previous idea to create individual, linked domains for a multi-level space.
Remote compute for spatial reasoning: We implemented a remote server capable of running raycasting, pathfinding, and exporting 2D maps.
Apple Vision Pro: We created our first AVP integration demo of a fully immersive Cactus space.
Reconstruction flow: Instead of manually copying DMT scans to a computer and running python scripts, we completed development of an automated reconstruction flow:
Film the domain in DMT and upload the recordings to the domain server.
Trigger reconstruction in DMT.
After reconstruction is complete, the point cloud (cleaner than ever before!) is sent back to the domain server and becomes viewable in DMT.
Scene reconstruction breakthrough: In another remarkable breakthrough, we figured out how to do scene reconstruction in a way that doesn't rely on QR detection or centralized feature matching. This means that (after further R&D next year) almost any physical space can become a domain without having to put QR codes all over it first.
Robot in domain: We gave the robot below access to the domain and created a real time visualization of it moving within the domain. It was able to follow the navmeshes without ever needing to map the environment for itself. The robot was the result of a long process of development and iteration by our robotics team this year. Read more about their work here.
DePIN map: When we first implemented map saturation rewards in September, there were 75 active hexes in the Relay network and we hadn’t launched domain servers yet. Now we have 235 and 233 active hexes on the Relay and Domain server maps, respectively. That’s over 3x for the Relay network!
Rounding out the year
The rate of change has been incredibly fast and has only accelerated throughout the year. We can’t wait to share the developments that we are working on right now. We are building some incredible, game-changing applications for spatial computing. Functionality that builds on, automates and streamlines what we built this year.
We are building something with real world utility. The pilots we ran this year have proven that. Stay tuned for exciting partnership and commercial announcements early next year.
If you think 2024 was an incredible year at Auki, you haven’t seen anything yet.
About Auki Labs
Auki is building the posemesh, a decentralized machine perception network for the next 100 billion people, devices and AI on Earth and beyond. The posemesh is an external and collaborative sense of space that machines and AI can use to understand the physical world.
Our mission is to improve civilization’s intercognitive capacity; our ability to think, experience and solve problems together with each other and AI. The greatest way to extend human reach is to collaborate with others. We are building consciousness-expanding technology to reduce the friction of communication and bridge minds.
The Posemesh is an open-source protocol that powers a decentralized, blockchain-based spatial computing network.
The Posemesh is designed for a future where spatial computing is both collaborative and privacy-preserving. It limits any organization's surveillance capabilities and encourages sovereign ownership of private maps of personal and public spaces.
The decentralization also offers a competitive advantage, especially in shared spatial computing sessions, AR for example, where low latency is crucial. The posemesh is the next step in the decentralization movement, responding as an antidote to the growing power of big tech.
The Posemesh has tasked Auki Labs with developing the software infrastructure of the posemesh.
.png)
.gif){kind=spacer}
.png)
.png)
.png)
