The method: measuring every letter
A keyboard makes thousands of small bets a day on what you meant. This is how we decide which bets it may make.
Every time you lift your finger, a keyboard bets on a word. Every time you press Space, it bets that what you typed is what you meant, or that it is not. A good keyboard is one whose bets are right far more often than wrong, and whose wrong bets cost you as little as possible. That is a question of measurement, and Armath Input Intelligence is built around answering it honestly.
The rule comes before the result
Before a change is measured, we write down what it must achieve and what it must not break, and we record that rule in the project’s history. Only then do we run the numbers. If the change fails its own rule, it does not ship, however good it looks.
That is less obvious than it sounds. It is easy to try ten variants and keep the one that happened to score best, or to move the finish line once you see where the runner stopped. Writing the rule first takes those temptations away.
An idea is not an improvement until it has passed a test it could have failed.
Texts we read only once
We develop on one set of Armenian texts and confirm on another that is kept aside. The confirmation texts, sentences from the Armenian treebanks of the Universal Dependencies project, are read once for a given question, after every choice has been made, and we record that they have been used.
The same care goes into training. Before Armath’s language model learned from Armenian web text, we checked how much of that text overlaps with our test sentences, and we report results both with and without the overlapping cases. When we found that an old word list contained the vocabulary of our own test texts, we said so, replaced the list and measured everything again.