MusicXML files should be available along with the scores!

[MusikChan]

MusicXML is like midi except that when you import it into Sibelius or Finale it will look like a score instead of a huge jumble. Importing midi files often results in getting a huge jumble which is unpleasant to look at. Many of us like to study scores and listen at the same time and having a MusicXML file makes this incredibly easy.

MusicXML is relatively new and there is not a lot of repertoire that has been made into MusicXML files (to my knowledge Bach's WTC has not even been made into MusicXML). I believe that if IMSLP began such an initiative it would popularize MusicXML and more people would begin to convert works into this format.

Also this is my first post on these forums. Hello everyone

[jemiller226]

If I ever get to the point of starting my dream project (getting typeset parts for all the scores here, which I do realize is precisely as ridiculous as it sounds), this will sort of take care of itself in a way. They'll be in PDF format, but the save file will be somewhere, of course, and if that format is eventually accepted here, all the better.

[KGill]

For reference, please see a few of the earlier threads we've had discussing this topic: viewtopic.php?f=3&t=2417 viewtopic.php?f=12&t=1412 viewtopic.php?f=7&t=977
The basic gist is that every score on the site is in PDF format for reasons of both administrative/organizational convenience and maximum utility to our users, the vast majority of whom are able to easily open PDF files without a second thought (I can even open them in my browser without an Adobe plugin). This is extremely unlikely to change in the near future, though there were plans within the last month to create a separate repository for source files only (specifically to be started to house some of WIMA's files). The chances of source files being accepted to the main IMSLP site, however, are at this point slim to none.

[musicxml]

All the threads referenced are 2 to 3 years old. A lot has changed in the MusicXML world in the meantime:

1) The format is now supported by over 150 programs, including nearly every notation editor in widespread use, whether proprietary or open source. Many readers are available on Windows, Mac, LInux, and iOS. See http://www.recordare.com/musicxml/community/software for a full list of software.

2) The compressed .mxl format introduced in MusicXML 2.0 takes care of the file size problem of the original uncompressed format. It also makes it easier to open MusicXML files by having a unique file extension that distinguishes them from other types of XML files.

3) Formatting support has greatly improved in MusicXML 2.0 and 3.0, as well as in the supporting software from programs, so the quality of the import has improved. PDFs of course still have the most accurate formatting, but offer no interactive capabilities for playback, transposition, analysis, and the like.

Keeping PDF as mandatory makes a lot of sense, but I am not seeing a huge administrative cost to adding compressed MusicXML .mxl files as a second file type. It seems that offering an interactive format would indeed provide maximum utility to IMSLP users. Since these files need to be created from new typesets, they will of course be dwarfed by scans at first. However, that may well change after several years.

With the maturation of MusicXML technology, could IMSLP please reconsider the possibility of offering MusicXML files together with PDF files?

[kalliwoda]

In pondering this request, please also consider the program PDFtoMusic that quite successfully converts PDF-files to MusicXML - if the PDF files were written from a Music program (rather than scanned).

Alas, my hope that the program would be able to recognize colliding and hidden elements in some of the more useless typesets on IMSLP were dashed....

[pml]

Thanks Michael, for your update on developments in the MusicXML world. We’ve discussed this around and around, and the usual issues that arise are as follows, and ancora, these result in some questions which I hope MusixChan would be prepared to answer:

1. Licensing. Unlike scans of old publications, MusicXML files would have to be licensed under Creative Commons licences to be hosted at all at IMSLP. Who would be submitting these? If they are purely mechanical transfers (e.g. using PDFtoMusic as Kalliwoda mentioned) then why would we be bothering to host them at all?

2. Reliability. Who is going to review these scores to check the contents to ensure they are correctly described and are a fair rendering of the composition? Like MIDI files (and some low-quality typesets, which we are attempting to eliminate from IMSLP) they would presumably have to be regarded as “caveat emptor”.

3. Stability. Which versions of MusicXML should be supported? Which particular version should be treated as being the minimum supportable? How would version control work — would an inaccurate score in a supported MusicXML version be preferred over an accurate score in an unsupported early version?

4. What users want. There’s strong evidence (contra a very few noisy commenters on these forums) that PDFs are useful to the vast majority of the site’s users. How does adding optional MusicXML files help IMSLP? How many users would gain benefit from them versus the additional administration work required to support it?

Regards, Philip

[MusikChan]

@PML

I am not a legal or web design expert, but I will do my best. I know that I use IMSLP quite often and I know how XML support would benefit me and users like me.

1. Licensing.

The upload page would have a disclaimer that uploading any original XML files would automatically imply that it is for use under creative commons for users of imslp. Again I am not a legal expert so Im not sure of the best way to go about this.

>Who would be submitting these?

If this were to take effect I am sure that many dedicated users would begin to convert scores. Right now there is no concentrated place to host them. If the number 1 online score library supported them then people would want that score library to have those files. I would definitely be interested in converting some scores for public use.

2. Reliability.

>Who is going to review these scores to check the contents to ensure they are correctly described and are a fair rendering of the composition?

Use the voting system like the one that is used for the pdfs, when a file receives a low score or a complaint it can be checked.

3. Stability.

Im not able to speak on this, hopefully musicxml can tell us more. I use Sibelius and I have not had any problems opening any of the MusicXML files that I have encountered.

4. What users want.

>PDFs are useful to the vast majority of the site’s users.

I totally agree with you there.

>How many users would gain benefit from them versus the additional administration work required to support it?

Maybe not a huge amount at first, but the internet was scoffed at at first too. I believe that as time marches on that more and more users will want the benefits of an interactive score.

>How does adding optional MusicXML files help IMSLP?

Are we talking about helping a single website or the music community as a whole? I believe the bigger picture is to benefit the music community. Interactive scores can help in so many ways. For one, people who cannot read sheet music can watch a score in Sibelius or Finale and it will begin to make sense to them. I like to mute out a voice in a composition and then sing along with it for ear training. Soloists will no longer need to rely on accompanists every time they want to practice along with the music. These are just a few uses that immediately spring to mind. The benefits of interactive scores are limitless and in the future they will be in use. Imagine theory textbooks viewable on ipads with XML examples, this is not so far beyond our reach. XML or a similar format will become mainstream and IMSLP could be the platform by which it is delivered.

[jemiller226]

Just playing devil's advocate here, but who checks the user-submitted, user-created typesets for accuracy now? I fail to see how this would be any different, in either that way or the license issue.

[daphnis]

@Jen, I do, for one, and I know several others do as well. Accuracy is one thing, but this is supplemented by application general notation practice. Both need to be followed.

And a general comment for all, MusicXML, along with source typeset files, are certainly encouraged, however they need to be attached to the PDF generated from them. Even if you don't really want to submit a PDF, if you're going to contribute an XML file, it's a fairly trivial task to generate a PDF from it, then attach and submit.

Philip makes valid points, especially with #2. Relying purely on a voting system is not acceptable as this is not how typesets are approved/disapproved. To offer the same sort of fairness to users who spend time tweaking their typesets to ensure they adhere to our policy, reviewers would probably have to import the XML file to see how the data is described. This naturally opens up several holes including 1.) this would probably limit the number of reviewers that are equipped to handle them and 2.) the translation would need a different set of criteria. The imported result would have to be taken with a grain of salt since it can't be expected to maintain or achieve a similar layout as a typeset.

[steltz]

Re: user ratings. I have come across scores on IMSLP that had very high ratings -- 8 out of 10 -- for scans that were unusable due to staff lines that were faded or broken in crucial parts of the pages. And if only one unexperienced or biased person rates a file, it will have that rating regardless of whether it is an accurate assessment of the file. The same will hold true for user ratings of typesets. User ratings are definitely not reliable.

[MusikChan]

@steltz

>I have come across scores on IMSLP that had very high ratings -- 8 out of 10 -- for scans that were unusable due to staff lines that were faded or broken in crucial parts of the pages.

well if there is no effective system of quality control over pdf files, then why should we worry about extra quality control on xml files?

[daphnis]

Because QC on PDF files is not necessarily the same as QC on user-created files. The quality of PDF files produced from scanned, "engraved" music is largely a function of the skill with which they were digitized--not constructed. With typeset PDFs or XMLs (which originate as typesets of some sort), the quality is totally in the user's hands. Quality here meaning more "quality of construction" and not "quality of capture". It's basically an "apples to oranges" scenario.

[MusikChan]

While Im not sure how this makes a bad pdf more excusable than the possibility of a bad xml file..

Another solution would be to require that xml files be created from a pdf thats already hosted on the site. The time it would take to ensure quality could be as short as just checking a minute of the work while following the pdf score. You could listen to 1 minute, follow the score, and if it appears that the author has done a quality job in that 1 minute then you can just go ahead and approve it.

You could do this on a limited basis at first, only accepting popular works by the major composers for a trial period.

[steltz]

Some of the scans that aren't so good are also of historically important manuscripts, so there is an argument for having a scan even if it isn't so good, but I don't think we want to extend that argument to absolutely everything.

Quality control on a site that had over 100,000 files before the WIMA merger was difficult, and WIMA had about 65,000 scores. We do need to make sure that whatever QC policy is in place is actually practical for those of us who do the work.

[musicxml]

For issue 3, I would recommend supporting compressed MusicXML 2.0 files as the minimum version. They offer small file sizes and widespread support. Earlier MusicXML versions don't allow compression and would not be practical for IMSLP hosting. Compressed MusicXML 3.0 files could also be supported. For the other issues in 1 to 3, I don't see any practical difference between the issues involving MusicXML files and those involving new (vs. scanned) typesets. For issue 4, looking at download statistics from other libraries like CPDL that offer source files may help answer this question.

Perhaps your experience is different, but my impression is that PDF attachments have usability issues for many people, both in creating and reading PDF files. Having a separate MXL file optionally available in addition to a PDF file seems like a more musician-friendly approach.