Remove duplicate files from Mac without affecting Traktor? Help!

[Adrian Luvdup]

Hi,
I was wondering if anybody could help me out with a particular problem as I’ve always found the advice on here great.

I have a Macbook with Traktor installed. There’s currently around 16,000 songs in my Track Collection.

The problem is, in my Itunes Music Library on my hard drive, thousands of the songs have been duplicated two, sometimes three times and I’m reaching capacity with my memory. If I could remove these duplicates I reckon I could free up absolute loads of memory.

I know I can purchase software to find duplicate music files and delete them. However, I don’t know which of these files are linked to my Traktor music collection and which one’s aren’t.

If I deleted all the duplicates on my hard drive I imagine that when I re-opened Traktor I would be faced with thousands of exclamation marks next to tracks. If I selected “relocate” from the drop down menu for the collection, would it actually find them again? What worries me is that the copies have a 1 or 2 added on the end of the file name, so I imagine Traktor wouldn’t find them. I could do it manually but for 1000’s of Tracks this would take an eternity.

This wouldn't be a problem for the main collection - I could just wipe it out and re-install the entire contents of my music library once the duplicates had been deleted - but I think it would be a nightmare for my playlists, of which I have many, some of a few thousand songs.

Ideally, it would be great if I could find out quickly which of my music files were linked to Traktor, and then be able to delete the rest - but short of looking at the path name for every track (again, this would take forever) individually, I can’t see how.

Does anyone have any ideas?

By the way, I do have a large external hard drive with a copy of my music library on it so I guess I could test anything on there first and be able to link Traktor back to the one on my Macbook's hard drive if anything went disastrously wrong.

[Ancipital]

The best way, the safest way.. is to back EVERYTHING up and just go for it- then you have nothing to lose barring a little restore time.

[Karlos Santos]

Adrian first thing to do is backup your music and your T3 folder if you gonna do any big collection changes (you know that anyway).

Then run Consitency Checker in Traktor and see how many missing tracks you have and Relocate them. This will find anything that is actually still there, any tracks deleted previously or altered will appear with exclamation marks. This way you wont eventually delete anything you dont want to. Then your Traktor collection is up to date. Close Traktor to update your Collection.nml

Secondly in iTunes find all your Duplicates (in View). Delete everything you dont want. Ive got some tracks with 7 entries, absolutely pointless.
When you start Traktor again run Consistency Checker again and you will see loads of missing tracks. These will be the ones you have just deleted from iTunes. Select, 'Delete Missing Tracks'. This will delete all tracks from the collection with '!' next to them.

You should be good to go then.

btw i think i used to come see you DJ at FEEL in Preston ?

[Adrian Luvdup]

Hi Karlos,

Thanks for your reply - yes, me and Mark were the residents at feel in Preston for quite a while. We did the more funky House set in the back room many years ago!

Cheers for the advice, can I just run through some of your points?


Adrian first thing to do is backup your music and your T3 folder if you gonna do any big collection changes (you know that anyway).

No problem, already have the music saved on an external hard drive. I’ll make sure I save the Traktor folders too.

Then run Consitency Checker in Traktor and see how many missing tracks you have and Relocate them. This will find anything that is actually still there, any tracks deleted previously or altered will appear with exclamation marks.

At the moment when I run consistency check, it shows no missing tracks as I’ve pretty much tidied up the Traktor interface side of things so there’s no duplicates showing in my collection in Traktor even though there are loads of them on my hard drive.

This way you wont eventually delete anything you dont want to. Then your Traktor collection is up to date. Close Traktor to update your Collection.nml

Secondly in iTunes find all your Duplicates (in View). Delete everything you dont want. Ive got some tracks with 7 entries, absolutely pointless.

I’ve got no duplicates in my iTunes window at the moment as I removed them all to tidy it up as well but didn’t actually delete them in case I had problems with Traktor.
Maybe I could just to a full import of all my files again so all the duplicates are showing in iTunes. I could maybe uncheck the tracks that are in there at the moment so when I do the import, all the duplicates will be checked and I can delete them quicker?

When you start Traktor again run Consistency Checker again and you will see loads of missing tracks. These will be the ones you have just deleted from iTunes. Select Delete Missing Tracks. This will deleted all tracks from the collection with '!' next to them.

This is where I get confused! If some of the duplicates I delete in iTunes turn out to be tracks that Traktor is linked to, then the tracks will show up in Traktor with exclamation marks won’t they after a consistency check? Because the actual file that the track was linked to has been deleted, I’ll then need to relocate the track in Traktor to the alternative track that still remains linked to my iTunes library. In fact, what I did was rip or download purchase as many of my tracks as possible at the highest quality for use in Traktor, but then copied them in iTunes as just 128kbit and then deleted the higher quality track from my iTunes library (but not the actual file as that was linked to Traktor) so I could store more songs on my iPod. Therefore, there will be absolutely thousands like this. If only you could use Traktor to move files to Trash like iTunes, I think my problem would be solved.

For example, if I have a folder called “A Guy Called Gerald”, in my iTunes music folder with 3 copies of the same track in it (“Voodoo_Ray.mp3” would be the original copy and iTunes would rename any further copies along the lines of “Voodoo_Ray1.mp3, and “Voodoo_Ray2.mp3”), if the original copy was linked to my iTunes collection and one of the duplicates was linked to my Traktor collection, then I would have to relocate it in Traktor if I had deleted it at the previous stage. Would I have to do this manually for every track as it has a different file name to the one originally linked to Traktor? If it was the same name I could just choose relocate for the entire collection but as the file name is now going to be slightly different in many cases I’d have to click relocate on each individual track in Traktor, find the folder in iTunes music, select the track and click ok for every one of them. This is the scary bit because there’s no rhyme or reason to which tracks are linked to Traktor – some will be the original files, others will be the second or even third copy! This could mean I might have to do this for anything up to 16,000 times.

Does any of this make any sense, it’s so hard to try and articulate and what I’ve just written above just looks like waffle but I hope it explains my worries.

Thanks again for your help, any further advice would be much appreciated!

[Karlos Santos]

Im a bit confused, but we can get this sorted.
First thing , Do you have a separate iTunes library and a different Traktor library Folder? Most people just use the iTunes Music Folder as their only Music Folder Library.
If you do, are all your Traktor tracks in your iTunes library as well?

If you import your Traktor Music folder into itunes will that then give you duplicates. If so you can do that , then go through the duplicates and import the iTunes library into Traktor to give you one Music Library.



If you have Imported your iTunes Folder as your default music folder then ALL your iTunes tracks are 'linked' to Traktor not just the originals.

iTunes should show all tracks so you should have duplicates.
If you do a search on 'Voodoo Ray' do you not get 3 results?
You can then choose to see the tracks in Explorer.

If you delete a track from iTunes and select to Remove From Your iTunes Library then it will be permanently deleted and Traktor will not be able to Relocate it. Traktors Collection is dependant on the default Music Folder. In your case is that your iTunes Folder?

Bit confused as to your setup?

[Karlos Santos]

Adrian Luvdup wrote: All the songs files are in one place on my hard drive, I think it’s the folder located at “music/ itunes/ itunes music” off the top of my head.

That makes things easier. But will impact on your Playlists. If you delete duplicates they will disappear from your playlists.

But if Traktors Default Music Folder is iTunes (which it will be) then ALL your tracks will be linked to your Traktor Collection not just some. This is where i think you are getting confused. Everything in iTunes should be in Traktor. If you have 3 versions of Voodoo Ray in iTunes you will have 3 in Traktor.

How do you make your Playlists in iTunes or Traktor cus if you are deleting Duplicates in iTunes you will lose the tracks in iTunes playlists but they will show '!' in Traktor Playlists and you can then relocate another version of the missing track. Unfortunately you cant export Traktor playlists back into iTunes unless you have Traktor 2.6. If you do have that version i can show you how to do that .

Confused? Yeah , its a pain. And some of this will take time.
But when you get chance check your iTunes and Traktor collections and see what correlates . Then you can start to arrange them better.

(I see Chris Hunter a lot, I was the resident DJ at his dads pub The Adelphi for years, i also used to be res at Promo in that same room you used to do at FEEL. Im now the resident at 53Degrees which is the new S.U. building, small world eh.)

[Adrian Luvdup]

Hi Karlos,

It is a small world! I'm off to Chorley next month as well, you ever played the Park Hall Night Club there?

The thing is, if I was to export my iTunes folder to Traktor tonight, then all the extra copies would appear in Traktor, but I've basically deleted them from the Traktor library as I've gone along whilst keeping them on my hard drive.

There are rumours of a new function for Traktor being the abilty to delete tracks not just from Traktor but from the hard drive. I could then give all my tracks already in Traktor a star rating, import the entire contents of my hard drive to Traktor, delete all the tracks without a rating as these would be extra copies and then they'd be permanently deleted. I could then delete the entire iTunes library and import the new, clean and tidy contents of my hard drive onto it (I don't mind sacrificing my iTunes playlists).

If this feature was on the horizon, I would wait for it as it would make things so much easier


One last thought, could I maybe write tags to the tracks in Traktor which would show up in iTunes, so if I was then to put all my music in my iTunes library, I could see which ones I had tagged in Traktor and delete the rest?

[Karlos Santos]

Adrian Luvdup wrote: The thing is, if I was to export my iTunes folder to Traktor tonight, then all the extra copies would appear in Traktor, but I've basically deleted them from the Traktor library as I've gone along whilst keeping them on my hard drive.

Ok i get it now you have maintained the Traktor Folder independently from iTunes

Adrian Luvdup wrote: There are rumours of a new function for Traktor being the abilty to delete tracks not just from Traktor but from the hard drive. I could then give all my tracks already in Traktor a star rating, import the entire contents of my hard drive to Traktor, delete all the tracks without a rating as these would be extra copies and then they'd be permanently deleted. I could then delete the entire iTunes library and import the new, clean and tidy contents of my hard drive onto it (I don't mind sacrificing my iTunes playlists). If this feature was on the horizon, I would wait for it as it would make things so much easier

That would work just fine however i think this is a Traktor Pro feature not T3. Will check that out.

Adrian Luvdup wrote: One last thought, could I maybe write tags to the tracks in Traktor which would show up in iTunes, so if I was then to put all my music in my iTunes library, I could see which ones I had tagged in Traktor and delete the rest?

Would have to try that, not sure if iTunes reads the Star Rating from Traktor , maybe the Comment Field would work. Ill give it a go when i get chance.

Sorry for the short answer , on my way out but ill check in angain and we will get this sorted.

Never played at Park Hall but been to many nutty nites there

[Adrian Luvdup]

Well Karlos,
Last night I thought I'd just give it a go.

First of all I gave all the tracks in my iTunes a 3-star rating, then I added the entire music folder to my iTunes. Then I deleted the 1000 or so extra tracks that had appeared without the star rating.

I then ran a shareware program called iDupe which sorted out my duplicates even further.

I then opened Traktor and did a consistancy check, deleted everything in the track collection but did a full import from my music folder again. This seemed to sort out my track collection fine.

I then clicked on each playlist and selected "relocate". This took a while for each one, but remarkably nearly all the tracks with exclamation marks were found automatically (some playlists had nearly 400 tracks with exclamation marks). Only a dozen tracks were missing!

I now have a tidy music folder, iTunes library and Traktor library and it freed up 8g of memory.

How easy was that in the end?

Thanks for all your help and assistance with this.

Ade

[Karlos Santos]

Brilliant , glad it worked out for ya.
Anytime.

Karlos

[Rob909e]

Could I touch on this?

On my MBP I have the same song listed multiple times all over the HD. This is crazy! Last night inside TSP I Exported my entire collection to HD/Users/documents/Native Instruments/Traktor/Settings. I also Exported the same collection file to an external HD.

I'm not sure how this happened but inside The Traktor Folder, Looking in Finder, if I look in the "backup folder" I can see lots of collection.nml files, however, MY ENTIRE COLLECTION is listed mp3's, .wav's, EVERYTHING. If I back out into the Traktor folder then go into "History folder" only dated .nml files are listed, this okay. Back out into the Traktor folder then into the "Settings folder" My ENTIRE COLLECTION is listed again mps3 wavs and everything inside the settings folder are other folders which are supposed to be there: Default settings, midi mappings keyboard settings and all the .tsi files

So my collection is listed 2 times throughout the Traktor folder.

On my Desktop I have a folder "downloaded music" which has 221 songs and was last used in April 08 ? Where, why, how, and what this folder is liked to I will never know.....

On the Desktop I have a folder "Beatport Music" This folder is for all my Tracks Downloaded from beatport.com. I think this is the Default download folder for the Beatport downloaded.

HD/Users/Music/Itunes/Itunes Music has a lot mp3's and wav's as well.

Basically it's a cluster F*#K of music. I use mixed in key right after a song is downloaded, where ever that may be, than I import into traktor either by drag and drop or by Right clicking.

Mr. Santos, Can you help me ohhh Wise one?

So in a nutshell, Their are 5 even 6 copies sometimes of the same song scattered around the hard drive... Which copy's can I safely delete without losing cues, loops, mixed in key tag info inside Traktor?

[Karlos Santos]

Hmm, how many playlists do you have? How big is the collection?
The next version of traktor will enable you to delete from the hard drive within Traktor. What you could do is give all the tracks in your playists (therefore the ones with cues,loops etc) a 5 star rating.

Then sort your collection by track name and all your tracks will align and you can delete the duplicates that dont have a 5 start rating. That way no tracks in any Traktor playlist gets deleted.

Obviously with a large collection it will take ages but it takes as long to do it in iTunes using the Show Duplicates feature.

Any good?

[buggles]

go to your traktor collection in the traktor browser...list alphabetical by song name and at each instance of multiples, right click and show in finder, see what they are linked to, and delete the all but the one copy you want from the traktor collection.

make sure all your songs are accounted for in one folder, erase the other copies through the finder and be done with them.

i never use itunes. i analyze the key - write the tag (just the key info) - select all the files in the finder - drag and drop them to the collection. analyze in traktor and beatgird that sh*t. then i drag items into individual playlists from the collection in the browser, because if i drag them into the playlist from the finder and it's already in the collection, i'll wind up with 2 copies in my collection that are linked to the same file.

in prefs, i make sure traktor isn't automatically importing folders because i suspect it may be causing doubles to reappear in the collection - but don't take my word for it.

READ A BOOK! da dant DANT! (go ask your mother)

[Karlos Santos]

This will not work if he has made his playlists in Traktor.
Using your method there is nothing in the Finder that will tell him wether a file is used in a Traktor playlist or has any cues/loops etc so he could be deleting files that are in his playlists.

You clearly didnt read what his problem was.

And as for the line at the end of your post , what the f*ck is that meant to mean?

[Rob909e]

THANK GOD FOR TIME MACHINE!

I figured since I have everything backed up I could just go ahead and do some trial and error, however, it seems that my trials have produced all errors!

Once I purchased Mixed in key I found that Itunes was useless to me since you can't organize by "key." Once I download or rip a track into my MBP I use Mixed in key for key analysis. Then I import the tracks manually into TSP Track Collection. (File Management setting in TSP: "Show consistency check report on startup" is the only boxed checked.)

Importing an mp3 then the key is picked up when I analyze the track. Next. I Load the track into Deck A for beatgrid and BPM adjustments, Traktor is not to good at auto beat gridding DnB. Once I feel the track looks good I Click on the 'lock' so I know the track has been worked on. I repeat the process until all new imported tracks are finished.

Importing a wav the key is not picked up when I analyze the track. I then right click onto the track, click 'edit' and then manually key in: artist name, track name, genre, label, and key. I also make sure to have all boxes checked on the fields I want 'saved'. When I'm done editing the track info I click apply then ok. After that I just follow the same process for beatgrids and BPM as I do for mp3's

Wav files don't keep metadata but Traktor saves whatever info that I add to the track: Beatgrids, new correct bpm, cues, loops. ect.

What would the Ideal paths be for my track info, loops, BPM, and the track itself without having duplicates. Could I have a folder on my desktop that has every single song that I want in TSP then Delete ALL the other duplicates. Also, how does that affect the info stored already in TSP.

I've been at this for days!
---
Oh sorry.

My Total collection is about 1200 songs

I have 8 playlists in the playlist tree in which I use them all for different steps in preparation for a set.

thanks for the fast reply. I have a similar thread somewhere else... I'm gonna try to remove it.