logo
Main

Forums

Downloads

Unreal-Netiquette

Donate for Oldunreal:
Donate

borderline

Links to our wiki:
Wiki

Walkthrough

Links

Tutorials

Unreal Reference

Usermaps

borderline

Contact us:
Submit News
Page Index Toggle Pages: [1] 2 3 ... 5 Send TopicPrint
Very Hot Topic (More than 25 Replies) 11GB of extracted (?) cache files, (+ 30GB GigaCache released) (Read 8625 times)
Carbon
Senior Member
****
Offline


Tinfoilus Illuminatus

Posts: 287
Location: Living Room
Joined: Jan 15th, 2013
Gender: Male
11GB of extracted (?) cache files, (+ 30GB GigaCache released)
Mar 26th, 2016 at 7:15am
Print Post  
Hi everyone.

I always wondered what was in the large number of cache files I have on my hard drive, so I thought I would try to extract (is that the word?) them all. I added all of the files from Ultimate Cache (parts 1 ~ 5), Sickpup cache, Newbies cache and another 'Ultimate cache' (parts 1 ~ 8), totaling 13.6GB.

I then compiled all of the entries from the various cache.ini files that were in each archive together and set about extracting them all.

While I started with 13.6GB, I ended up with only 11GB of files. Not sure what happened, but I can only assume that many of the .UXX files had no reference in the .ini and were unable to be extracted.

Anyhow, I have uploaded what was extracted into 5 parts with Mega. It is a free account with a 10GB limit (daily, I assume?), so I hope that someone can get these and host them (if they are worth it).

I put all of the resulting files into the appropriately named folders (.unr files into a 'maps' folder, etc), then zipped it all up.

I cannot say if this is useful or not, nor can I say if all of the maps will work. I haven't tested them all (there are just over 2800 maps), but some of them are not available elsewhere (I did some Googling for some of the map names with 0 results) and may have some value to someone.

Here are the links (with the key) and again, I hope that someone with more bandwidth can grab these and host them. I can also share a BTSync folder as well, if desired.

https://mega.nz/#!M1R0GL6a!-hci_HDiNSMHhlkkZKERw6m28skUeKlewQs2Iy6ZVHc

https://mega.nz/#!Ylh2hSzJ!naTpeVaLBSB-D7RPKV1uMW-TeHMiSN4Y6iKOaYJ8kAw

https://mega.nz/#!ws52hLqZ!_1TIcwzrvkH5FI3o2uWR7nlg81GoOXlUgkGws9gxeWA

https://mega.nz/#!1lYW0KZK!lg4QcjjATNwJd5T5AgTBl_ntf0ltZmSL1hnUqGe-PAI

https://mega.nz/#!N0JgGIKS!1RHOaAFYx8AGWTjpvRIXKKLDxMiL83GHVlA3wLmAD3M

Update: A small thread hijack by Hyper (with permission)
Hyper's GigaCache project has finally be released. See http://www.hypercoop.tk for details or read further on this thread for the preceding discussion leading to the release.
« Last Edit: Mar 29th, 2016 at 8:02pm by Hyper » 
Reason: Added GigaCache release info. 
Back to top
 
IP Logged
 
gopostal
Betatester
Offline


Hack coder

Posts: 805
Joined: Jul 31st, 2008
Gender: Male
Re: 11GB of extracted (?) cache files
Reply #1 - Mar 26th, 2016 at 2:50pm
Print Post  
Carb, could we set up a Sync folder for this? Once I get the whole thing I'll create a torrent for it and add it to the 'Entire Skin' torrent I constantly host. That's really the best way to get this out there especially at this size. It's what a torrent really excels at doing.
  

I don't want to give the end away
but we're all going to die one day
Back to top
 
IP Logged
 
GEx
New Member
*
Offline


Deep cover GEcko!

Posts: 23
Location: Kaunas, Lithuania
Joined: Jul 31st, 2013
Gender: Male
Re: 11GB of extracted (?) cache files
Reply #2 - Mar 26th, 2016 at 3:51pm
Print Post  
Downloaded it, but it's giving me an unexpected end of archive (also the total of all 5 zip files is 4.57 GB). Undecided
  
Back to top
 
IP Logged
 
gopostal
Betatester
Offline


Hack coder

Posts: 805
Joined: Jul 31st, 2008
Gender: Male
Re: 11GB of extracted (?) cache files
Reply #3 - Mar 26th, 2016 at 4:19pm
Print Post  
I was able to download and unzip the entire thing Gex. It's up to you if you want to try again or not. I'm currently 7zipping the collection and then I'll post it as a torrent (if you want to wait). I'll post back as soon as I get done.

Edit: Here's the torrent
https://dl.dropboxusercontent.com/u/36642316/ExtractedCacheFiles.zip.torrent

I'll keep superseeding for a few days or until everyone gets on. The entire collection ultra compressed with 7z to 4.7GB. I ran through it all before compression and files looked good and intact. Thank you Carbon!
« Last Edit: Mar 26th, 2016 at 6:35pm by gopostal »  

I don't want to give the end away
but we're all going to die one day
Back to top
 
IP Logged
 
Carbon
Senior Member
****
Offline


Tinfoilus Illuminatus

Posts: 287
Location: Living Room
Joined: Jan 15th, 2013
Gender: Male
Re: 11GB of extracted (?) cache files
Reply #4 - Mar 26th, 2016 at 11:09pm
Print Post  
gopostal! I had hopes that you would stumble upon this thread. I hope also that all is well with you.

Thanks for putting the torrent together. I will d/l and seed it infinitely on my end; I probably should have just done that from the start.

  
Back to top
 
IP Logged
 
[]KAOS[]Casey
Developer Team
Betatester
Offline


nedm

Posts: 3113
Joined: Aug 7th, 2011
Gender: Male
Re: 11GB of extracted (?) cache files
Reply #5 - Mar 27th, 2016 at 1:22am
Print Post  
If only the fabled Hyped "GigaCache" would make an appearance like this. With a torrent.
  
Back to top
 
IP Logged
 
Carbon
Senior Member
****
Offline


Tinfoilus Illuminatus

Posts: 287
Location: Living Room
Joined: Jan 15th, 2013
Gender: Male
Re: 11GB of extracted (?) cache files
Reply #6 - Mar 27th, 2016 at 2:00am
Print Post  
The only 'gigacache' talk I have ever heard of - past & present - has been done by Hyper. Indeed, most current Google searches include him (and a number from OU) and mention of a 10GB+ pack.

As I said, I based this collection off of 4 cache archives, totaling 13.6GB of which I recovered only 11GB.

For me, the weak link (2.6GB unrecovered) seems to be the cache.ini files being incomplete. I copied and pasted all entries from all .ini files into one document, placed that into the master folder where I extracted all of the cache archives then just let it run. It produced numerous errors regarding files being without reference. I assumed this is a deficiency in the .ini; there are .UXX files that have no .ini referent or perhaps double entries in the .ini. However, I am not sure about this being the correct interpretation of the error, so unless someone can interpret it - do as I did to check this out - then this collection is as exhaustive as I can see it being. At least from my end. I'm sure many people have old, obscure cache files that could be included to round it out more.

So, maybe if we all pool our cached files, compile a master cache.ini file, then perhaps we can make the fabled 'gigacache' for ourselves.

Maybe Hyper could ring in on the topic?

EDIT: Made another torrent for the heck of it; gopostals is unseeded right now.

https://www.dropbox.com/s/6imo5u58zqt7p1d/Extracted%20Cache%20Files.zip.torrent?...
  
Back to top
 
IP Logged
 
[]KAOS[]Casey
Developer Team
Betatester
Offline


nedm

Posts: 3113
Joined: Aug 7th, 2011
Gender: Male
Re: 11GB of extracted (?) cache files
Reply #7 - Mar 27th, 2016 at 2:35am
Print Post  
Gigacache was started long before cache.ini existed. The purpose of gigacache nowadays is to find old & lost mods/textures/maps/music that were once hosted on servers. It would be trivial for me to write a native mod to attempt to load packages and figure out whats in them, I could very easily tell what a .utx vs a .umx and a .unr look like.

Problem is, there's no way to know their original names. At that point.. it might be possible to make a seperate native mod that tries to load dependencies until they work, and renaming packages until i've sorted most of the ones with dependencies. It's totally doable.

I just need the files.






edit:

To clarify, I could figure out names of packages that are a dependency of other packages.

For instance, if a map was named "BestMapEver.unr"† and it depended on "BestTexturesEver.utx", and I loaded them as .uxx files wit hgarbage names, I could figure out the name for BestTexturesEver.utx since it is a dependency of BestMapEver.unr, based on what BestMapEver wants out of BestTexturesEver.

BestMapEver's name can never be known without a cache.ini entry. Similarly, some file extensions can never be known -- any unreal package can store any type of information. a .UTX can have a map, and music, but contain no textures. The naming is arbitrary, but the convention is to stuff music exclusively into a .umx, maps exclusively into a .unr, and so on.
  
Back to top
 
IP Logged
 
Hyper
Board Moderator
Betatester
*****
Offline


It's Unreal.

Posts: 2896
Joined: Oct 11th, 2002
Re: 11GB of extracted (?) cache files
Reply #8 - Mar 27th, 2016 at 3:14am
Print Post  
Oh, the old GigaCache. It just keeps growing for some reason.† Cheesy

Just a huge pile of treasury junk or junky treasure. Including bytehacked files, gazillions of UTeamFixes and EnhandedDM's. Never to be used again. 99.9% unlisted in cache.ini. No, I'm still not going to upload the monster. End of experiment. Still, quite a homage to the productivity of the Unreal community over all those years.
  
Back to top
WWW  
IP Logged
 
[]KAOS[]Casey
Developer Team
Betatester
Offline


nedm

Posts: 3113
Joined: Aug 7th, 2011
Gender: Male
Re: 11GB of extracted (?) cache files
Reply #9 - Mar 27th, 2016 at 3:15am
Print Post  
As expected. I am disappointed, once again. You call it junk, I call it treasure.
  
Back to top
 
IP Logged
 
Hyper
Board Moderator
Betatester
*****
Offline


It's Unreal.

Posts: 2896
Joined: Oct 11th, 2002
Re: 11GB of extracted (?) cache files
Reply #10 - Mar 27th, 2016 at 3:23am
Print Post  
By the way, why do most of the cache files have the number "4" at the 9th character of the file name?
  
Back to top
WWW  
IP Logged
 
[]KAOS[]Casey
Developer Team
Betatester
Offline


nedm

Posts: 3113
Joined: Aug 7th, 2011
Gender: Male
Re: 11GB of extracted (?) cache files
Reply #11 - Mar 27th, 2016 at 3:24am
Print Post  
I'm not going to even bother investigating that after the post you just made. I owe you nothing.
  
Back to top
 
IP Logged
 
Carbon
Senior Member
****
Offline


Tinfoilus Illuminatus

Posts: 287
Location: Living Room
Joined: Jan 15th, 2013
Gender: Male
Re: 11GB of extracted (?) cache files
Reply #12 - Mar 27th, 2016 at 4:06am
Print Post  
Hyper wrote on Mar 27th, 2016 at 3:14am:
99.9% unlisted in cache.ini.


Well that pretty much closes the idea in any practical sense. Too bad though; I'm sure there were some things worth preserving in there!

Thanks for the reply Hyper. At least the legend of the great and mysterious gigacache is closed! Smiley
  
Back to top
 
IP Logged
 
[]KAOS[]Casey
Developer Team
Betatester
Offline


nedm

Posts: 3113
Joined: Aug 7th, 2011
Gender: Male
Re: 11GB of extracted (?) cache files
Reply #13 - Mar 27th, 2016 at 4:27am
Print Post  
Doesn't matter if its not practical. There are ways to make it practical. I can recover all maps with a 100% certainty, and all music with a 100% certainty. Just can't know their original names.
  
Back to top
 
IP Logged
 
[]KAOS[]Casey
Developer Team
Betatester
Offline


nedm

Posts: 3113
Joined: Aug 7th, 2011
Gender: Male
Re: 11GB of extracted (?) cache files
Reply #14 - Mar 27th, 2016 at 4:28am
Print Post  
You say ďJust a huge pile of treasury junk or junky treasure.Ē

I say a lost history. A treasure trove of knowledge and mods long forgotten. What if that contains the first set of mods that ever inspired me to mod? What if it contains the mods that inspired other people to start modding? My whole career started way back in 2000/2001 because of these things when I was eleven years old. Because of this, I owe the community everything I have, really. Even if in reality, I owe no one nothing because all of the knowledge I learned was mostly self taught and no one directly taught me, cache ripping was what got me started on creating new mods. Aura, in particular, actually.

It all started as copypasting here and there, but because I was so young at the time I really was not able to comprehend what I was taking so I couldnít just write my own code. So I took blocks of code from other mods that did things that I wanted to do and eventually through enough repetition and willpower I was able to code (albeit really badly). These mods were from the cache. The only mods I really downloaded myself or were sent were from koopa or other various things like u4e but u4e didnít have features I desperately wanted to adapt.

I have lost mods myself that no one else seems to have. You might actually have them. In fact, actually, I guess I do owe you something, because looking back on it, http://www.oldunreal.com/cgi-bin/yabb2/YaBB.pl?num=1413052201/3#3 you helped me recover one lost mod of mine (though it was made by dots). I guess Iíll look for that cache thing later just because of that. Oops. Guess I was wrong.

That doesnít stop the fact that if someone does have a lost mod or a map, they may not ever be able to find it. I knew the exact class name that was probably unique so it was trivial to find. I have the knowledge, among with a few others, to make use of this enormous cache to find old files and mods to just give people the ability to attempt to find lost treasure. Things lost in time, things lost because of hard drive failures, or obscurity. Sure, some things probably donít deserve to see the light of day, like you said with the billions of copies of EDM and stuff. Tons of mods made by drake, elementmaster, skaarj hybrid, peoplesí names who I have simply forgotten because it was over ten years ago now. People I used to consider bad or enemies, I donít care. Their mods shouldnít be lost to time.

You, Hyper. Youíre choosing to not allow the community to have itís treasure.

You have not once cited a specific, tangible reason other than ďI wonít do that.Ē to PMs to me, emails to others, posts to others. I bet you even if we mailed a USB blu-ray drive, a stack of blu-rays, and enough money to post the blurays back to whoever wanted it and you wouldnít do it. Why?

Why?...

If I had to make a grand guess, you actually donít care. Thatís what happens when you donít give a good reason. People have to make assumptions. Assumptions are bad. Youíre also guilty of assumptions, like assuming all custom consoles are cheating, even though thatís laughable since cheating is stupid in cooperative play. Maybe people were making custom consoles clientside to test bugs? How would you know externally. You wouldnít, I get that. I myself have made a custom console for 227h/i that re-enables set/get and such for testing clientside because I need it for testing. But because of you and others custom console blocking I had to create an undetectable (without unreal integrity) console hack via C++. Bravo, I guess, for contributing to helping me learn more.

Back to not caring: Guess who does care?

I do. This topic does. We do. But not you, thatís obvious with the little amount of information we have. We know you keep your home server up 24/7, you could easily upload 10kb/s 24/7. Yeah, sure, itíd take about 70-80 days of straight uploading, but that data could be uploaded to somewhere/someone with a goddamn massive pipe and weíd all be able to reap in the file in no time. Especially if you use torrent software (Itís legal. Iím sure you know this.), this couldnít be too big an issue. 70-80 days is nothing for what this contains. Absolutely nothing. (Also, your cache is not compressed either, Iím sure the compression ratio may be pretty good however.)

70-80 days. Iíve probably put in more hours debugging unreal, creating test/crash cases, looking up code in unrealscript and the headers trying to break things. Iíve spent so much countless time pouring into unreal over years and iíve never asked for a single god damn thing. Iíve hosted download/backup sites for dots, turboman, and mirrors unrealsp. Hosted files for anyone who asked. I even did this when I made damn near nothing money wise OR actually nothing a month. 227 wouldnít be nearly as stable as it is without as much time and effort that I put into it, not to discount others. Dots put in an incredible amount of time too, and now masterkent and others who I really canít remember their names right now because Iím writing this massive wall of text. I never asked for money, I didnít really want recognition. Helping unreal be better was my goal. This is still on my mind. By not uploading this, youíre actually hampering the community in my mind.

So now, Hyper.

I will ask the first thing Iíve really ever asked that was of any importance to me or others. Not that I havenít already, but:

Upload the gigacache. Please.

  
Back to top
 
IP Logged
 
Page Index Toggle Pages: [1] 2 3 ... 5
Send TopicPrint
Bookmarks: del.icio.us Digg Facebook Google Google+ Linked in reddit StumbleUpon Twitter Yahoo