HSCALE - Horizontal Scaling for MySQL

HSCALE is a plugin written for MySQL Proxy which allows you to transparently split up tables into multiple tables called partitions. In later versions you will be able to put each partition on a different MySQL server.

Download the current version NOW! Version 0.2 released on 2008-05-06. To get started read on Configuration and Running. The current development status can be found here. Report any issues to our JIRA Issue Tracker.

Find more information on MySQL Proxy and HSCALE and the reasons why here: Presentation Slides: Introduction to HSCALE

News and Blog Posts

pero on anything » hscale
Current project status - HSCALE A lot of people keep asking me about the status of HSCALE so I thought it is best to just write about it. Since I am waiting for the next GPL release of MySQL Proxy I concentrate on other things, both project-related and not project-related. Continuous integration and test strategy enhancements First of all there is a [...]
LuaSQL fetches results about 15% faster than MySQL Proxy? While evaluating LuaSQL as backend connection replacement I came across this. I did a quick performance test using mysqlslap and it showed that just reading and copying the result can be significantly faster with LuaSQL. Benchmark details What I did was just sending the query to the backend and building up a new result-set in LUA. This [...]
Version 0.3 of HSCALE is almost in the door After working on build and test improvements (for example incorporating lualint and LuaCov) as well as other lua “side-projects” (i.e. Log4LUA) we are running towards HSCALE 0.3. The focus of the forthcoming version 0.3 of HSCALE is Dictionary Based Partition Lookup. Using this partition lookup module lets you take full control over how your partitions are [...]
More fun with LUA - Introducing Log4LUA UPDATE Log4LUA 0.2 released. Go to the project page. After the dust about backend connection handling in MySQL Proxy had settled, I begun working on other parts of HSCALE again. (I’ll return to the backend handling later after talking to Jan Kneschke about their plans.) One of the issues that bothered me the most was logging. Until [...]
MySQL Proxy vs. HSCALE Recently I added the first code to support multiple MySQL backends in HSCALE (see svn.hscale.org). As a “side note” I have to thank Giuseppe Maxia for MySQL Sandbox which made multi server testing a bliss! While coding this I started to feel that writing HSCALE on top of MySQL Proxy is no more as easy and [...]
HSCALE 0.2 released and new project web page The main focus of version 0.2 was to improve handling of almost all of SQL. So now you can issue DESC TABLE tbl_name or RENAME TABLE tbl_name TO another_tbl_name on a partitioned table and you get correct results for SHOW TABLES etc. Other statements are rejected and we settled down for the feature set we want [...]
Update: Benchmark HSCALE with MySQL Proxy 0.7.0 (svn) against 0.6.1 Earlier today I posted these benchmark results testing HSCALE and MySQL Proxy performance. As Jan Kneschke (the author of MySQL Proxy) pointed out there are quite some improvements in the current development version (svn trunk). So I gave revision 369 a try. Tests were all the same as mentioned in my previous post. And indeed we see [...]
Benchmark MySQL Proxy and HSCALE As part of developing HSCALE, a partitioning / sharding solution, I set up a benchmark test suite. I made it scripted and thus repeatable to monitor the progress and performance regressions during the development. Test Suite The test suite uses mysqlslap to benchmark the overhead of MySQL Proxy itself in real life scenario as well as the [...]
Presentation Slides: Introduction to HSCALE No, these slides are not fresh from the User Conference in Santa Clara… Today, I held a presentation in front of all developers and support engineers of our technical department about database partitioning, MySQL Proxy, HSCALE and the progress we are making. Download the presentation slides here.
HSCALE 0.1 released - Partitioning Using MySQL Proxy As written here and here I’ve been working on a MySQL Proxy Lua module that transparently splits up tables into multiple partitions and rewriting all queries to go to the right partition. I finally got everything together to release a 0.1 version. Go on and download, try and read more about HSCALE 0.1. All this started out [...]

Application Based Partitioning / "Sharding"

Application based partitioning means that your split up your data logically and rewrite your application to select the right piece of data (i.e. partition) at any given time. More on application based partitioning. Read some more about what could be done with HSCALE.

HSCALE helps in application based partitioning. Using the MySQL Proxy it sits between your application and the database server. Whenever an sql statement is sent to the server HSCALE analyzes it to find out whether a partitioned table is used. It then tries to find out which partition the sql statement should go to.

What About Partitioning Inside MySQL 5.1 and Higher?

Partitioning provided by the MySQL server is a whole different thing. In the most simply way: It manages how your data is stored internally. You still have a 1,5 billion row table with almost all the disadvantages: you cannot alter the table easily and you cannot put different partitions on different MySQL servers.