Support

Forums

Contact Me

WikiPedia architecture and scalability


This PDF is a lot more technical, but reveal some challenges WikiPedia is facing in order to maintain its infrastructure/response time.
WikiPedia is simply the biggest multilingual free-content encyclopedia on the Internet. Over 7 million articles in over 200 languages, and still growing.


[...] Started as Perl CGI script running on single server in 2001, site has grown into distributed platform, containing multiple technologies, all of them open. The principle of openness
forced all operation to use free & open-source software only. Having commercial alternatives out of question, WikiPedia had the challenging task to build efficient platform of freely
available components. [...]

One more time, worth reading if you are into web development, performance and scalability. It seems that lighttpd is more and more used for serving static files (html, images, js, css, pdf ...) instead of the venerable Apache

If you have still some new fresh neuron to burn, you can read the  Overall system architecture and more HERE
The most important news here is that Wikipedia currently uses APC, so I choose the right PHP cache ;-)

Ebay architecture scalability

At SD Forum 2006, Randy Shoup and Dan Pritchett, both with eBay, gave a presentation on eBay's architecture. Pritchett subsequently posted his presentation slides in his blog, The eBay Architecture.
Predictably, the presentation contained a few inspiring statistics, such as:
  • 15,000 application instances running in eight data centers
  • 212,000,000 registered users
  • 1 billion page views per day
  • 26 billion SQL queries and updates per day
  • Over 2 petabytes of data
  • $1,590 worth of goods traded per second
  • Over 1 billion photos
  • 7 languages
  • 99.94% uptime

Other stats in the presentation related to development process and features, such as:

  • Over 300 new features released each quarter
  • Over 100,000 lines of code released every two weeks

  • "According to the presentation, the goal of eBay's current architecture is to handle an additional ten-fold increase in traffic, something eBay expects to reach within a few short years. Another architecture objective is to be able to handle peak loads, and for components to gracefully degrade under unusual load or in the case of system failures." read more  HERE

     

    YouTube Scalability Talk

    Kyle Cordes’s blog post on the “YouTube Scalability Talk” It’s definitely worth reading!

    Speed up your web pages with YSlow

    YSlow analyzes web pages and tells you why they're slow based on the rules for high performance web sites. YSlow is a Firefox add-on integrated with the popular Firebug web development tool. YSlow gives you:

    • Performance report card
    • HTTP/HTML summary
    • List of components in the page
    • Tools including JSLint

    Download HERE
    Visit also the homepage of the team behind this tool: Yahoo!'s Exceptional Performance Team

    You should not take too much time to guess that I am currently optimizing my homepage :-)
    Be careful, running this tool more than 5 times make my Firefox crash ;-)

    Some statistics:
    Some useful links
    Be careful! a lot of host get pretty low metering (just test your favorite internet site), but are still very fast -> this tools do not tell You their infrastructure: memcache server, multiple db server, multiple file server for static content and so on.

    Donations

    Thank You for supporting my work