IMSLP Forums

A new IMSLP regional server has been activated in Hong Kong. For the time being, this server is intended to act as a backup for the European and the Canadian servers only in the case of major problems with them. No traffic will be automatically routed to this server unless specific problems occur.

Users experiencing any problems to access the European or the Canadian servers (due to server outage or congestion, network issues, etc.) can easily reach the HK server as follows:

- for files normally available on the European server, replace "imslp.eu" with "imslp.hk" or "imslp.asia" in the URL of the file;

- for files normally available on the Canadian server, replace "petruccilibrary.ca" with "imslp.hk" or "imslp.asia" in the URL of the file.

The new server is available thanks to the donations to support the existing European and Canadian servers made by IMSLP users. The funds collected seem now sufficient to activate and maintain this backup server in order to ensure continuity of operations in the case of problems. The IMSLP-ASIA server is managed by Project Leonardo Limited, the NZ company already operating as the internet service provider for the European and the Canadian server.

Choralia great to finally see this being set up it is definitely needed! Couple questions:

1. I assume one reason behind this server being set up is in case the TPP is passed in a few months? Canada might be moving to the Life + 70 model which is very worrisome right now.

2. Depending on where you live (say in the US or the EU) and you were trying to access a piece on one of those servers, is there going to be any slowdown in downloading a piece because of the distance to the server since it might take longer to download vs. if the pieces were on their original servers? Have you noticed any slowdown when testing these servers?

Thanks!

Sallen112 wrote:one reason behind this server being set up is in case the TPP is passed in a few months?

The primary reason is disaster recovery: should a complete outage of the regional European or Canadian servers occur, no one score would be lost, and normal service operations could be restored in a short time. Hong Kong was selected for geographical separation (it is very unlikely that a disaster in Canada or in Europe also involves Hong Kong), as well as because it is a 50 pma country where copyright terms are not expected to change, at least in the near future.

Sallen112 wrote:is there going to be any slowdown in downloading a piece because of the distance to the server since it might take longer to download vs. if the pieces were on their original servers?

Yes: distance is distance and it does not help. The time to download a sample file from each regional server is regularly measured every 15 minutes from a monitoring server located in the US. As an example, these are the download times recorded in the last 24 hours (the file size is approximately 5 MB):

- Canada: minimum 0.9 seconds; average 5.7 seconds; maximum 11.7 seconds
- Europe: minimum 2.6 seconds; average 5.9 seconds; maximum 12.2 seconds
- Hong Kong: minimum 3.4 seconds; average 10.6 seconds; maximum 77.8 seconds

If the monitoring server were located in Far East rather than in US the results would be probably different, with the shortest time from HK instead.

Max

For question 2, yeah that is what I thought. I am surprised that the maximum time for the Hong Kong one is so much longer than the others whereas the minimum time is about 1 second longer. Do you know why the maximum is so long for the Hong Kong? I assume the distance (from the US) has to cross the Pacific Ocean (for me anyways) and for you (I remember you said you live in Europe) could just cross over the Asia Continent to get to both the Asian and the Hong Kong servers.

Sallen112 wrote:Do you know why the maximum is so long for the Hong Kong?

No, I don't, but I'm not much surprised. On a road network, the longer is your jorney, the larger is the probability that there is at least one road segment on your route that is congested by traffic. When there is no traffic congestion, the travel time is shorter and pretty predictable, even on very long routes. When one road segment is congested instead, the travel time can be whatever. It's the same on the internet: when the whole route is free, the download time is short and repeatable. When some parts of the route are congested, the download time can potentially increase to any value. The longer is the path, the larger is the probability that at least one congested segment exists along the route.

In Europe and North America the number of internet backbones is probably so large that, if one road segment is congested, it is quite likely that an alternative route is available to reach the destination without a very large delay. Perhaps the number of internet backbones in Far East is not as large as in Europe and North America, and therefore there is a larger probability that a congested path cannot be resolved through an alternative route without adding a significant delay. This is my interpretation.

Max

I have been in discussion with Carolus about the TPP IP chapter (the whole text has just been released online) and I think we should consider thinking about preparing for moving all server operations for the Life + 50 files onto this server in the case Canada does pass the TPP in their country next year (which would extend copyright from 50 to 70 after the death of the author, so it would be aligned with the EU server copyright length). It will be too bad that we can't have this server (if this agreement is passed) just as a temporary backup for outages but this agreement with the 12 pacific nations is a vital one for IMSLP's future of where the life + 50 server will be held and used. I hate to see this happen but our lawmakers make the decisions here, there is not much we can do besides yelling at them (metaphorically speaking) to vote down the agreement!

Hey Choralia about the outage on the main site that happened a couple days ago. Since the outage happened on the main server in Canada, to prevent something like this happening again, would it be possible to prevent these types of things happening again to redirect the site immediately over to the Hong Kong Server (and users) in case the site goes out in Canada once again? This way the site is still functional while the Canadian one is under maintenance (and vice versa) Then when the server is back up in Canada, can the traffic and the site be moved back to the Canadian one? The laws are very similar between Hong Kong and Canada so I don't see why we can't have the server over there activated when that outage occurs on the main server!

Geographical diversity is one of the main strategies to ensure service continuity. However, it has some drawbacks:

- cost, as multiple servers are required (minimum two), and each of them must be able to manage the whole data processing and traffic load;

- complexity, as muliple database instances are required, and they have to be constantly aligned. When one database instance is down, re-aligning it to the others at the end of the outage may cause some data inconsistency issues.

Currently the regional server in Hong Kong is sized to backup the regional servers in Europe and in Canada, not the main IMSLP server. Regional servers contain no database replica and receive a small percentage of the whole IMSLP traffic, so they are shared servers, quite inexpensive. The regional server in Hong Kong is also a shared server, for the time being. Upgrading it to a dedicated server similar to the main IMSLP server would significantly increase the operational costs.

At CPDL we have adopted a "clustered" strategy that is pretty cost-effective: there is a main server (www.cpdl.org) that is only used by registered users (i.e., users are automatically routed to it after logging-in), and four mirrors (www0.cpdl.org, www1.cpdl.org, www2.cpdl.org, www3.cpdl.org) that are read-only and handle the traffic generated by anonymous users (i.e., users who have not logged-in). Mirrors are located in different areas of the US, and their number can be increased as needed. Anonymous users are randomly routed to mirrors in order to balance the whole traffic load, and the percentage of traffic handled by each mirror can be set at will. The main server is a dedicated server (actually, a Virtual Private Server - VPS), while all mirrors are inexpensive shared servers. All mirrors are aligned to the main server on a daily basis, so they are pretty up-to-date.

When a mirror is down, anonymous traffic can be easily re-distributed to the other mirrors by just setting to zero the percentage of traffic routed to the faulty mirror. If the main server is down, registered users can anyway access mirrors as anonymous users. New works cannot be uploaded when the main server is down (mirrors are read-only, so nothing can be edited or added on them), however everything already online remains available to everybody through the mirrors. At CPDL we are using this architecture since 2009, and it's pretty robust.

Cloud technology may be another alternative. It's similar to the "clustered" approach we use at CPDL, however the distribution of traffic and processing load to multiple servers is automatically done by the cloud architecture. The drawback is that one cannot closely control the geographical distribution of resources, and this may be tricky if, for example, some files are stored on servers located in different copyright areas (e.g., in the US rather than in Canada). Costs should be also evaluated.

Anyway, I'm ready to support any initiatives aimed at improving IMSLP's service continuity. This subject may be possibly discussed with the Project Petrucci folks.

Max

I don't think cloud technology would really work with the web structure of this site because of the copyright laws between each country. Like you said, it would be harder to control where files would be located on servers and would be all over the place (basically a mess) around the world, thus violating regional copyright laws. The files have to be controlled on where the files are exactly located. I am not a engineer at all but this is something that should really be thought about for security/loss of data on servers, going forward.

Would there be a cheap alternative way to mirror (not cloud or a dedicated server) the main site on the Hong Kong server? I am just worried about the loss of data mainly if something say more outages happen on multiple servers around the world at the same time! The servers are running 24/7 so outages are bound to happen!

Just another thought, what if we could mirror the main site but just make it readable only where you can still browse around but you can't edit anything in it, until the main dedicated servers back in Canada are up? This way we can still access content on the site for Users to still be able to download files but we cannot change anything with webpages. Also I imagine then uploading files during that time wouldn't work either? My idea is that we should have the site accessible while the main dedicated servers are back online.

All you mention is definitely feasible, and largely corresponds with the approach we use on CPDL. Please note that all IMSLP files are mirrored already on multiple servers to prevent bandwidth bottlenecks. I'm not sure about database replication or backup on remote sites, though. This has to be discussed with Feldmahler and/or Carolus. I can just confirm my availability to support any solutions with my IT skills (not great skills indeed, but anyway much better than my talents as a musician ).

Max