Stampin' Up!

Problem Definition

Stampin' Up is a large online retailer with local demonstrators. Their customers look forward to new releases and when new products are released the number of visitors per second to the website increases exponentially!

During some of these releases, as new content was being published, some of the web servers would start to experience outages which would cause increased demand on the other web servers. The problem had been isolated to new content being published while servers were under increased load.

REDEVTION was engaged to determine why the publishing of new content was causing problems only under heavy load, come up with a solution, and then determine a plan for future improvements to their deployment process.

Process

Initially, REDEVTION was provided with the client's code to get familiar with how the website was set up and to determine if there were any issues within the code. After an examination of the code, it was determined that the code was clean and that the website was set up properly. A few suspect areas were found but would need to be tested within a test environment to have access to a setup similar to production.

​

REDEVTION traveled to the client's location to work side-by-side with their developers. A war room was established and access was given to various team members to help with troubleshooting. An attempt was made to replicate the issue but was unsuccessful due to not being able to generate a sufficient amount of load on the website.

The tactics were changed to instead test various theories at a lower scale to determine their effects. The initial theory was that this was tied to a setting in the configuration files that determines how many times a recompile of pages can take place before the website is restarted. This had been a cause of problems with other clients and was suspected to be the issue here as well.

REDEVTION set up a small test environment and used various profiling tools to discover how changes to the content on a Sitefinity page are affected by the configuration setting. Through these tests, we were able to see that a content change produced several recompiles. When another visitor would request that page it would increase the number of recompiles. So under increased load, the number of recompiles would be reached quickly and would cause the application to restart. While the application was restarting it would continue to receive requests which would cause the website to crash.

We then performed additional testing and analysis to determine the effects of increasing the setting number of recompiles on the website. After sufficiently showing the client what the issue was, we were able to provide some solutions for the client to move forward with.

Solution