How do I know the concurrency of my highly scalable solution?

The team was excited. The demo went well. Our platform was appreciated and they agreed to use it. Great news indeed!

“We will use it from next week. Will it hold well when all users log in on Monday morning?” A common concern, based on past experience.

The team had done a good job of understanding the scalability requirements, designed an architecture that will scale well. However, during the initial phase, everyone was focused on functionality. Performance was yet to be tested.

As I was reviewing, I found one thing missing – concurrency requirements.

How many users can be active at the same time?

How many requests will simultaneously reach the server?

This is an essential factor for capacity planning. Too less will mean dropped requests and too much will mean higher costs.

There is no correct answer. Concurrency keeps changing based on external events, time of the day, user behavior, etc. It can be predicted only with visibility into past data.

This is where instrumentation comes to help. Build desired counters in the system, record, monitor, and notify them regularly. Allow for the tuning of what to check, what to record, and what to notify. You are going to need regular monitoring and tweaking for the best efficiency.

Do you have the counters built in your product?