We looked at many different options, including hiring an overseas team, building our own team, and hiring a firm to help. We decided to hire a firm in Chicago to help us develop version one of the software (starting in July 2007), but they were unable …

@tmprojects Very happy that the post helped.

@MasterMaharishi Thanks so much for those kind words, Scott. They truly mean a great deal. The past few weeks have been very taxing on us. They’ve involved a tremendous amount of work and lots of pressure to work out the issues in a timely manner. But the encouraging words from our community have kept us going and have truly shown us why we work so hard and why it’s so important for us to continue this path together.

@Randy Wise words. Our database servers are top notch (we run Raid 1 OS and Raid 10 DB) dual dual Xeon quad processors and plenty of memory. Sadly, the issue isn’t the hardware (although solid state could certainly help). Our database structure, as a result of the CMS that we elected to use back in 2007, does not scale well (it is not a relational database). As a result, the queries to the DB are large and tend to lock the DB tables on writes. This is an incredibly inefficient process and is the single biggest cause of our problems.

In reworking our application and moving to Python/Django, we’ve rewritten everything from scratch and reworked our model. Once we are able to complete the code and push the new site, these types of problems will (hopefully) be a thing of the past. For now, we’re trying to find ways we can create some breathing room so that we can complete refactoring. This includes working with the configurations of all our servers, including the DB servers.

Thanks again for the offer to assist…if we can’t figure this out, just might take you up on it.

@tmprojects Very happy that the post helped.

@MasterMaharishi Thanks so much for those kind words, Scott. They truly mean a great deal. The past few weeks have been very taxing on us. They’ve involved a tremendous amount of work and lots of pressure to work out the issues in a timely manner. But the encouraging words from our community have kept us going and have truly shown us why we work so hard and why it’s so important for us to continue this path together.

@Randy Wise words. Our database servers are top notch (we run Raid 1 OS and Raid 10 DB) dual dual Xeon quad processors and plenty of memory. Sadly, the issue isn’t the hardware (although solid state could certainly help). Our database structure, as a result of the CMS that we elected to use back in 2007, does not scale well (it is not a relational database). As a result, the queries to the DB are large and tend to lock the DB tables on writes. This is an incredibly inefficient process and is the single biggest cause of our problems.

In reworking our application and moving to Python/Django, we’ve rewritten everything from scratch and reworked our model. Once we are able to complete the code and push the new site, these types of problems will (hopefully) be a thing of the past. For now, we’re trying to find ways we can create some breathing room so that we can complete refactoring. This includes working with the configurations of all our servers, including the DB servers.

Thanks again for the offer to assist…if we can’t figure this out, just might take you up on it.

One area to explore is that of the I/O capability of your database servers, and the tuning of the web servers and underlying OS. I specialize in this area, and I can cite example after example where the underlying infrastructure was the issue – NOT the application. A poorly written app can hurt, but in 98% of the cases, no tuning of the app would help. Tuning can improve the capacity and the response times by over 10x – this will greatly improve the stability, user experience, and ROI of adding additional hardware.

You may also consider use of solid state storage as a means to dramatically lift I/O rates without changing your code.

I’m in Chicago. Contact me if I can be of assistance.

Randy

One area to explore is that of the I/O capability of your database servers, and the tuning of the web servers and underlying OS. I specialize in this area, and I can cite example after example where the underlying infrastructure was the issue – NOT the application. A poorly written app can hurt, but in 98% of the cases, no tuning of the app would help. Tuning can improve the capacity and the response times by over 10x – this will greatly improve the stability, user experience, and ROI of adding additional hardware.

You may also consider use of solid state storage as a means to dramatically lift I/O rates without changing your code.

I’m in Chicago. Contact me if I can be of assistance.

Randy

The letter you sent out and the detail you have gone into above, show me, and all of us creatives who use your site on a daily basis, that you are doing everything you can to keep us going.

Even with some band aids, the site is still the best in class!

Oh, and with regards to your comments about failure, cS could close tomorrow and would never be considered in that way!

Peace out and keep up the great work!

Scott