I will start from the end.
Over the last week Zapiens hasn’t been working quite the way it should, maybe you’ve seen your training interrupted or you’ve asked Zap something important.
My name is Iván Arrizabalaga and I’m the CTO at Zapiens. I’m responsible for you reading this post instead of some random stuff about cats or mindfulness, … you know.
Since I have now bothered you, I’d prefer not to waste your precious time so this is just to inform you that on 27th September during the night we will be updating the system with some changes that, hopefully, will put everything right on track again.
For those of you that are curious
During this last year, platform usage and features have been growing steadily. Luckily, or due to the care and attention that we pay to detail, bugs, outages, shutdowns and other such stuff haven’t been a common occurrence, or at least they have never been a real issue.
Unfortunately, during this last week our error rate multiplied by three.
This month, over a total of more than 3M requests we had an accumulated error rate of 0.09%. Not perfect, but the service has been working ok and all the errors have been minor stuff.
This week the error rate raised to 0.32% and, what is worse, some of our customers have found the «handmade» version of the Twitter whale, Linkedin cut cable, … argg.
Nothing weird or bad really. Our community is growing, more trainings are available, more questions to Zap, more requests, … sometimes our code wasn’t fast enough to resolve all these before the timeout came up.
What are we doing to solve it?
Basically we’re changing 3 things:
– Cleaning up the database, removing old, non required data to speed up some processes.
– Optimizing the process to generate the questions x day for users.
– Adding a series of transparent «retry» policies. Until today, since everything was working ok on first try, we didn’t have to consider this possibility. Quite simply, this has changed.
In conclusion …
If you have made it to here and you’re still reading – thanks! It is really important to create cool projects like Zapiens but it becomes meaningless if you can’t do it from transparency and responsability.