How TeskaLabs Helped O2 Improve Customer Satisfaction of eKasa Point-of-Sale (POS), the Most Successful POS Product on the Czech Market
In 2016, the Czech government introduced a new law requiring all businesses to report their sales figures and provide Electronic Evidence of Sales (EET). This law calls for the adoption of a more modern point-of-sale system or cash register that enables businesses to meet these new regulatory requirements. Over the next two years, the law will gradually impact upon more than three hundred thousand companies in the Czech Republic. O2, the largest integrated telecommunications provider in the Czech market, observed that many would need help complying with this law, while also maintaining data security and providing high quality customer support.
O2 decided to introduce their own mobile cash register, called eKasa. The aim was to create a professional POS solution that is unique in its uncompromised focus on quality, security and customer support. They created this new product during 2015, in the form of a mobile POS solution that runs on Android operating system.
While preparing this new product, O2 reached out to TeskaLabs and invited them to work together on this solution. TeskaLabs provides a technology solution for mobile applications called SeaCat. This solves questions of security, scalability and visibility, all of which are vital aspects of progressively scaling up professional applications such as large-scale POS solutions.
O2’s investment, along with the help of TeskaLabs, certainly paid off: eKasa is now the most successful POS solution available in the Czech Republic, and is used by more than a quarter of the entire Czech EET market.
eKasa is in high demand, and within a few days of launch there were already a large amount of customers using the service. The O2 team were very busy dealing with this rapid growth, and when in a situation like this, it is easy to overlook the occasional customer problem. However, it is precisely here that the difference between the good and the great lies: companies should always pay attention to detail, even when they are serving the masses. A single customer complaint will have much greater repercussions than ten satisfied customers who don’t comment on their experience.
Occasionally, O2 received random customer feedback about a slow connection to eKasa, which could prevent optimal usage of the service. Clearly, this was a serious issue that required appropriate attention.
Around the time of the eKasa launch, TeskaLabs were able to use SeaCat technology to detect unusual behaviors at a network level in the communication between eKasa devices and O2’s servers. Issues occurring at such a low level are undetectable by common monitoring tools available to O2 operation teams. Luckily, though, SeaCat made all the difference. SeaCat was the only component that picked up on this issue, because it collects a wide variety of operational telemetry data from many different aspects of a mobile application, including communication, performance, availability, and so on. Not only does SeaCat function as an extensive cybersecurity tool, but it also provides detailed input for data analysis tools. This meant that SeaCat was able to offer management and technical staff a unique insight into the eKasa application.
SeaCat discovered anomalies at the TCP level, the fourth layer of the ISO/OSI network mode via an audit logging component. SeaCat registered incoming TCP RST packets from eKasa POS immediately after it established TCP connection. In basic terms, this meant that the connection from the POS terminal was being immediately terminated, and eKasa wasn’t able to communicate with its servers to work properly. During the intensive post-launch period, the number of RST packets grew by a significant amount.
TeskaLabs carried out the investigation along with key figures from O2's Security Expert Center, IT data networks, and mobile networks. It was challenging to figure out the problem because at the time of the investigation, the RST packet anomaly was not linked to user complaints.
There was one more issue at hand: only a small percentage of eKasa devices were being affected, and that introduced a challenging yet very common issue when operating large-scale distributed applications like this POS app. You may observe random errors in some POS devices, but the chance that you will see the same error on your testing device is virtually zero. In this specific case, we calculated that it would take more than 50 consecutive days of testing to obtain a 90% chance of the issue appearing on an eKasa test device. Put simply, this meant that we had no way of observing the problem from a user’s perspective.
To work around this problem, TeskaLabs reconstructed the POS problematic state based on SeaCat telemetry data and audit logs, without having physical access to the affected devices. By doing this, we finally understood what was happening. Knowing the nature of the issue, we were able to bisect the problem and identify the component that was causing the issue. Bisection is a commonly used technique to isolate issues. It quickly became obvious that the anomaly was not on the POS app side, but somewhere deep inside O2’s mobile network.
Thanks to TeskaLabs' technology and proactive approach to problem solving, the issue was fixed before performance further impacted upon the user experience and satisfaction, and O2 were able to continue with the successful launch period.
Now every customer can use SeaCat-enabled eKasa without facing any network errors. SeaCat technology guarantees a smooth user experience - it measures everything, analyzes each client, and alerts administrators when it detects the slightest sign of wrongdoing.
If you’d like to get a true assessment of the security of your POS system and its backend, ask us about our Security Audit. Alternatively, see our POS system management solution to know how we can help you build and operate your POS system in a secure and reliable manner.
You Might Be Interested in Reading These Articles
SeaCat and OpenSSL Heartbleed Bug
After almost two and a half year we hope that the Heartbleed remains in the past. It is not true, unfortunately. Now we have proof that a security vulnerability remains with us for a long time, maybe almost forever even when there exist patches and fixes. The Internet is a battlefield among the good, the bad, and the ugly. Who has better attacking or defending technology wins.
Published on December 20, 2016
White box vs. Black box penetration testing
When it comes to hacking, there are many technical aspects that can be difficult to grasp without an extensive background in the field. One of the most common sources of confusion is the comparison between black box penetration testing and white box penetration testing.
Published on January 15, 2019
The Security Vulnerability That Puts Millions of Application Backends at Risk. Yours Included
FoxGlove Security researchers published a serious vulnerability that can put millions of application backend, including mobile backend, at risk. Mobile applications use the same web-app technology for their backends, thus suffer the same vulnerability. Mobile application servers are inherently insecure because they consist of extensive stacks of software. Each piece can contain risky zero-day vulnerabilities.
Published on December 15, 2015