Sibley Project List of remaining links

[icactus]

I have it generate 1000 titles and then I go over them while it works on the next 1000, so I don't have a full list of all the titles yet. If you want a batch of 1000 filtered links with titles to play around with I can send those to you. I'll send you the full list once it finishes either way.

[pml]

Hi icactus,

I was more suggesting that I could think of a use or two for the full unfiltered list (15,000!) with titles, and am happy manipulating the full list.

Cheers, P.

[icactus]

i have 4000 titles now (2000 of which were filtered earlier today). I'll just generate the full list of titles first before i start filtering it so I can send it to you sooner. I'll also have to go back and generate titles for the first 4000 that are already posted because I hadn't designed the title program yet.

[icactus]

I permanently removed the bad links so I don't have a way to get those back without starting over.

[pml]

No probs. I think it is worth noting that some of the links are “bad”, though not on the current set of pages. P.

[Carolus]

A few points to keep in mind about Sibley items, in case anyone was not aware:

1. There are a number of items which have been renamed (by me, among others) due to the fact that I changed the file-configuration. Sibley, especially in the earlier days, split up works like opera vocal scores into more-or-less even 50-page segments. For a number of these, I either a) merged the segments into a single file; or b) split a large score into its designated musical sections likes acts or movements. Please do not replace any file of this nature with the Sibley files, as our configurations are more user-friendly.

2. Sibley has re-done the scans for some things. I think they assign a completely new file number for the re-done item.

3. There's also the Category for the Sibley Mirroring Project, which keeps a count of all the items which use the template.

[Eric]

the first item on the list (544) IMSLP already has, (Lutz Reverie - just the Scanner field used the impermanent URL, no longer valid, deprecated (and warned against by Sibley) form for the link to Sibley/URResearch instead of the permanent link or the template SibleyScan that we can try to change if even the "permanent" link proves impermanent (as may well happen) - I advise adding a step to the list generation to check for urresearch.rochester.edu-form scanner links as well as hdl.handle ones

[Eric]

No probs. I think it is worth noting that some of the links are “bad”, though not on the current set of pages. P.

actually, that's part of the problem- some of the Sibley Scanner URLs are "bad", so we... erm... not sure we're talking about the same thing. never mind

[icactus]

@Carolus: Good to know. It looks like there are only a few hundred Sibley pieces that are in our system but not tagged so (less than 10% of total Sibley links) so we shouldn't run into a huge amount of conflict. Unless they used the exact same name I don't have an easy way to find out which pieces were scanned a second time by Sibley but hopefully that's also a relatively small percentage of the total.

I just finished the last manual filter so the list of links is current as of today. I split it into 2 pages to facilitate load time. Maybe 3 pages would be better..... Time to start uploading!

[icactus]

Even though we have this list, I'm still going through each one to make sure it's not already uploaded in a different format and I'm adding the Sibley tag on the ones that were missing it yet already uploaded so that eventually we'll have a complete list.

[Carolus]

This a really excellent tool you've made, Icactus. It should be most useful in enabling us to have a near-perfect mirror of Sibley's site in place.

[icactus]

Thanks! We might even be farther along than we think - I've already found a Sibley's that have file duplicates from the MIT archive so I double tagged them for the record. Hopefully there are more than just a couple....

[jemiller226]

I've been in contact with Jim Farrington and Alice Carli at the Sibley library, and we're trying to work on a way to take care of the files listed as restricted access. Some of them are that way in error because of a glitch on their end, but they can't easily find them, and it hadn't occurred to them that, while IMSLP might be able to host those files legally, that nobody outside the library could get to them in the first place. Of course, some of them are that way for copyright reasons as well, and in some cases they've restricted access even for PD-US works if they're still under copyright in the EU.

At any rate, they've asked for a list of all the restricted access links. I implied heavily that it would be a long time coming but that we're working on it, and gave them a link to the project link list page to show them what I meant. Hopefully, being a newbie, I haven't completely overstepped my bounds, but it all started with a simple question about a work they had mislabeled and it just snowballed from there.

[icactus]

Thanks for your help, jemiller226! It gets lonely sometimes working through those links and it's nice to have some company! Like they told you, they probably don't have anyway to know which are restricted and which aren't, so I would just make a note on the link list page after whatever link you come across that says "restricted access" so we can eventually compile them and submit them to sibley to be "unlocked."

Also, there are no bounds to overstep, short of deleting the entire link list for no reason, so don't feel like you have to tip-toe around. You're just as much a contributor now as anyone else. Feel free to let me know if you run into any sibley problems in the future, and thanks for helping!

[jemiller226]

icactus, a number of the restricted files may not be able to be unlocked for copyright reasons, but would be perfectly fine in Canada. In that case, we're going to have to work together with them to find a workaround for how to get the files to a Canadian so they can upload legally.