Cross Platform Development at Blippar with C++

This week there has been quite a lot of interest in the AirBnb Medium article where they describe their use of and decision to drop React Native.

The article raised a lot of interesting points, several of which overlap with our experience at Blippar in using C++ for cross platform development. Seems as good a time as any to write an article outlining our own experience, for good and bad.

Early Days and motivation

Blippar is an augmented reality platform which is available on iOS, Android and previously Blackberry 10 & Windows Phone initially and then as a universal Windows app. Blippar was initially only available as a mobile app, now it is also an SDK which can be integrated into another app. The goal of the platform is to provide a consistent experience for developers who make the AR experiences or blipps. There is an ‘AR engine’ which is the brain of the operation coordinating computer vision, asset loading, execution and lifecycle of blipps.

In the early days the entire engine was written twice, once on iOS in Objective C and once on Android in Java. This was ok, for a time. There were significant logical differences and issues on each platform and was difficult work to keep the two in sync. Most new development happened on iOS (this continues today) and required careful synchronisation between the two teams to make sure things weren’t lost in translation. Even as a small team things were inevitably lost. We often ask candidates at interviews, how do you make sure the other platform is in sync with your platform, in terms of features, bugs etc.? The answers pretty much always are the same, we have a project/product manager, a JIRA board and just coordinate changes manually. This seems very curious, developers in general are thought to strive for Don’t Repeat Yourself (DRY) yet they seem to accept having this massive amount of duplicated code.

As the company grew there became demand for other platforms, Blackberry 10 (BB10) came first. Of course neither Objective C and Java works on BB10, it was time to change our architecture. Our only real cross platform option was to go to C++ otherwise we’d end up have everything written three times.

The conversion to C++ was a long process but not as bad as feared given Objective C’s grounding in C. Initially we only converted the main core of the code, there was still a great deal of platform specific duplication left unconverted, primarily due to time constraints. Some particularly challenging aspects of the conversion surrounded lifecycle of the different platforms. Android, iOS and Windows are all very different beasts, it took some time to resolve issues with threads, resources, rendering and more. For example, on iOS and Android these are posix based so it was possible to share thread implementations with pthreads. Windows however has different threading interfaces which did not operate in the same way as pthreads.

When Windows came along, significant effort went into bringing more and more of the core into shared C++ code, the cost of a feature having a lot of duplicated logic on each platform now became quite high.

The Present

Although BB10 and Windows have since been dropped as platforms at Blippar, C++ lives on 7 years later. The majority of the app and SDK code is now in C++ and most new features are either written in C++ or interact heavily with it. We still strive to keep as thin a platform-specific layer where it is reasonable to do so. UI is not a good fit for cross platform, and did not even try to model screens or layout in C++. Along the way we’ve learnt a lot, both good and bad, about choosing this route.

The Good

• Cross platform C++ is truly cross platform, it allowed us to quickly move to new platforms such as Windows. That’s such a huge thing for a startup that is looking to hit the maximum reach as possible, maybe less so now given mobile platforms are fairly settled. However if HoloLens or Magic Leap became a requirement in future we can get onto that platform without requiring an entire re-write of the code.
• Features come ‘for free’ It’s a really great feeling when someone spends a long time implementing a feature in C++ and it can come to another platform with minimal changes required. Depending on the feature it can even come entirely for free, simply with a git update.
• New C++ standards Recent changes to the C++ language standards in C++11 and C++14 have improved the quality of the codebase. New features are coming in, albeit slowly, such as threading, locking, smart pointers which means we can use pure C++ without relying on custom platform implementations of these features. C++ has come a long way from C99 ¹
• Performance The performance of C++ is unparalleled. For our app and SDK we have to do a lot of work with images for computer vision, these algorithms must be highly efficient. Implementing such algorithms in a language such as vanilla JS simply cannot be as fast as C++. If you add in vectorisation/SIMD instructions the speed benefits improve even further. The general speed of C++ for execution of ‘normal’ business logic we’ve found far superior to similar implementations in our codebase in Java.
• Productivity Our team is small, it has never been larger than 10 engineers and often far less than that. Yet we are able to maintain an app, SDK and continually add features at an impressive speed. C++ has facilitated this, albeit with some associated downsides illustrated below.

The Bad

• Atypical developer skills required Recruiting high quality developers with the desired experience has always been a challenge for us. Not only do we want a different set of skills for the mobile platform itself (AR is unlike most traditional apps), adding C++ into the equation reduces the pool of candidates even more.
  Getting C++ developers who have traditional desktop experience without mobile isn’t so bad.
  Getting mobile developers is also not difficult.
  But getting devs with a reasonable level of both is really not easy. Over the years we’ve attempted to split the work into just C++ and native-only and have the devs overlay when needed. This works to an extent but there is often a communication and understanding gap between the two. A specialist C++ developer is often unable to work easily on Android especially without a mobile developer sitting next to them for prolonged periods of time. In reality, spending time finding a good developer who has both native and C++ being worth it in the long run.
  We’ve found that a lot of mobile developers find C++ intimidating and view it as an arcane impossible-to-decipher language.
  C++ is not a trendy language, Swift and Kotlin are the poster children for Android and iOS, so why on earth would a mobile developer want to learn a dying language like C++?
• C++ as a language There are no two ways around it. C++ is a complex language, and getting more and more complex. After being so long without change, C++11 brought in some huge modifications to the language. The complexity with these new features such as copy elision and r-values added to the confusion, you really need to know what you’re doing to remain efficient. There are so many ways to write code (and opinions on how to do it), it’s easy to shoot yourself in the foot if not reasonably experienced. If you’re a C++ developer at some point you’ll have seen some clever template metaprogramming code and been completely befuddled by it. C++ maintains backwards compatibility between versions, this is an immense amount of baggage going all the way back to ~1985 and earlier with C compatibility. Sometimes it feels like it would be such a cleaner place if we could have a clean ‘from-scratch’ version of the language.
  The complexity of the language is really why there isn’t a big market for developers who are cross platform gurus, it is a full time job to stay on top of each in it’s own right. People choose to become an expert in something they see a future in, there’s a lot more work in pure native mobile than C++; the choice for most is easy.
• Lack of built in functionality Even with the recent C++ language improvements there are still some glaring omissions when compared with other languages/platforms. No networking, decent file system APIs, database support, image loading, antiquated include system, lack of standard decent dependency management tools are a few that come to mind.
  Before C++11 there wasn’t even any standard locking/mutex and threading. We had to implement many of these using custom platform implementations, which while not especially tricky, required testing and introduced potential subtle platform differences.
  In this respect C++ is still a quite young language, C++20 will bring in some of these which is nice to see. Of course, the omissions are somewhat understandable, if a feature goes into the language it needs to be implemented by compiler vendors, and that has massive implications for a whole variety of platforms not just Android and iOS. C++ is in everything, any change in C++ affects millions of applications.
• Maintenance overhead Outside of the C++ code itself there’s a great deal of maintenance that typical mobile engineers just don’t need to contend with. Makefiles, compiler settings, architecture differences and building third party libraries are amongst the headaches that having C++ brings. Android is a real pain, with hours lost hunting for makefiles online for some library, googling cryptic compiler template errors or trying to figure out obscure linker issues. As a language it’s generally a drag and not pleasant to maintain.
• Platform Differences There is still no silver bullet to have the exact same code running on all platforms. There are differences in compilers, performance, memory layout and app lifecycles. With C++ the situation is fairly good but there are still issues that arise with seemingly standard C++ code that has different behaviour on the platforms. Android requires more third party libraries to fill in some gaps where iOS has built in libraries, these cause differences in behaviour.
  Furthermore, there are differences between what the platform itself provides to C++ developers. For example, iOS provides some nice high level APIs with CoreGraphics for decoding images. But Android and Windows do not (in C/C++), instead it required us to integrate of libjpeg-turbo and libpng along with several up/downscaling algorithms. The same code takes around a few lines of code on iOS. The Native Development Kit (NDK) on Android has much fewer C/C++ libraries than iOS, this is mostly due to the fact that Objective C is a superset of C.
• Compilation Times and debugging Compilation times in C++ can be very long, especially if the code is template-heavy. This can be a drag on productivity. Sometimes if changing a particular shared header file it may kick off a 5 minute build only for you to realise that you made a syntax error of some sort.
  Debugging C++ is not trivial either. The C++ standard library is heavily used and very difficult to debug. Different platforms provide different debugging facilities, Android Studio still lags behind Xcode in how well it supports just simple C++ debugging with breakpoints etc. In release, the debugger cannot really be trusted at all so debugging sessions can boil down to print statements.
• Platform Integration Objective C++ is a dream for using with C++. You can mix and match Objective C with C++ and everything plays nicely, just rename a file from .m to .mm and away you go ². On Android however you have the Java Native Interface (JNI) to communicate from Java to C++ or vice versa. It’s fairly horrible and very easy to get wrong, especially if you have a heavily multi-threaded app like Blippar. We adopted SWIG to greatly reduce boilerplate JNI code but SWIG is complex in itself (could merit it’s own post). The journey to get to where we are was a long one with much StackOverflowing. Now though all code in Java is ‘pure’ Java because of SWIG, native functions are hidden away in the auto-generated wrapper code. Some devs see SWIG as some sort of voodoo or magic and are a bit intimidated by it. However like C++, the benefits of SWIG far outweigh the negatives for us.

The Future

Although there is seemingly more negatives to positives above, the reality is that without using C++ Blippar would not have been able to scale and grow as it did. It allowed us to develop faster, have consistent performance and behaviour between platforms which has been critical to the success of the platform.
Back to React Native, C++ has a similar end-goal but requires significantly more investment both initially and on a continual basis. React Native wasn’t around when we started our journey, if it was would we have used it? Most likely not, it just doesn’t suit our needs as an AR company or provide the features we need.

It seems now more than ever iOS and Android are diverging from one another, being a cross platform provider is becoming more and more difficult. There isn’t any perfect cross platform solution right now. The best solution would be able to communicate directly with the native platform APIs, Xamarin comes closest to this in terms of completeness but it is still not working directly with the platform. There is no clear winner in terms of a language, JS seems to be winning the race at present but not because it is the ‘best’ but because it has the largest developer pool. Flutter is interesting but feel that emulating the native UI is never going to be right. DART is quite interesting as a language but also brings a learning curve that JS does not have, it is early days for that framework.

From Blippar’s point of view, there still isn’t really anything that can take C++’s place as a high performant cross platform language. There are a number of promising suitors, Rust in particular looks very interesting and will be keeping an eye on it.

Footnotes

1. Depending on who you ask, they may deem that C++ has gone backwards in terms of complexity and with the new features. Game developers generally favour C with a very small subset of C++ features.
2. It’s not all rosy, mixing Objective C with C++ does cause some issues. For example, OCMock cannot do partial mocks of classes that have C++ objects in the interfaces.
   Also any interface exposed to Swift cannot have any C++ exposed, it must be pure Objective C.