Page 1 of 1

Save unfinished cache files

Posted: Thu Apr 19, 2007 12:48 pm
by .:..:
I had this idea while seeing those temp cache files..
Would it be possible to make it save unfinished downloads in some directory and next time ur about to download the same file again it would continue downloading it. It's just so frustating when ur downloading a 10 MB file at 95 %, then server map change and you have to start over.. or is the idea too time consuming to make?

Re: Save unfinished cache files

Posted: Thu Apr 19, 2007 7:54 pm
by Shivaxi
omfg...this is a really good idea...i totally agree .:..: I dunno how this would work but it does seem possible :P

Re: Save unfinished cache files

Posted: Thu Apr 19, 2007 11:20 pm
by Bane
I'd assume that doing this would be relatively simple. While downloading the file, occasionally dump what has been downloaded so far into the file .tmp. When starting a download, check to see if this file exists. If so, open the file and resume the download from there. I've never tried doing anything with saving files in C++, but I can't imagine that it would be too difficult to implement ths.

Re: Save unfinished cache files

Posted: Fri Apr 20, 2007 7:14 am
by Smirftsch
didnt mess with these things yet, but i think the problem is maybe more the resume than to save it.

Re: Save unfinished cache files

Posted: Tue Apr 24, 2007 5:47 am
by TCP_Wolf
Yea sounds like requiring a few intelligent changes to both server and client. File handling itself is easy (relatively) and fast in C++. But you know things are ALWAYS more complicated than they first appear :-O

Re: Save unfinished cache files

Posted: Thu Apr 26, 2007 12:50 am
by DieHard SCWS
I often notice a Cache0000.tmp (Cache0001.tmp, Cache0002.tmp, etc.) files sitting in the root of my Unreal folder.


To me it always looked like a cache Temp like discussed here, but disfunctional. In other words, it looks like the Temp is created, but is indeed unable to resume. Unless you guys tell me those files are something different, but that is what i always thought........
.
.
.

Re: Save unfinished cache files

Posted: Thu Apr 26, 2007 6:00 am
by Smirftsch
no, you are right. It is what you think.
But on the other hand, it might be not necessary at all when the download is fixed. I tried it with havoks suggestion to set the tickrate of the server to 70- long time i ago i saw such a fast download. So there must be a way to fix that.

Re: Save unfinished cache files

Posted: Thu Apr 26, 2007 5:10 pm
by Bane
huh. I had no idea that's what those were for. I thought it was saved information from the pre-caching (not that I know what that is...). I just opened one of them in a hex editor and didn't see the GUID for it. If the GUID isn't in there (and it isn't in the file name), there's no way it could possibly resume. I'm guessing it's just a temp file that stores the information as it's received so it doesn't all have to stay in memory at once. The file format for these could probably be modified somewhat to allow recovering downloads.

Re: Save unfinished cache files

Posted: Thu Apr 26, 2007 10:06 pm
by .:..:
Well once I saw some map data in 1 of those temp files like "Brush103" "Nali1" etc...
So, I think it stores the currently finished downloads there while game is downloading, then when its finished it moves the file to cache folder and renames it. Maybe some unexpected shut down makes unreal not to delete them when download cancels (crash, exit?).

Re: Save unfinished cache files

Posted: Thu Apr 26, 2007 11:01 pm
by Bane
Actually, even pressing F10 causes the files to remain. Perhaps they had intended support for incomplete downloads at some point and just never finished it. Even if the files did persist after a crash when they should've been deleted, unreal could just clean them all out on next startup, like with purgecachedays

Re: Save unfinished cache files

Posted: Wed Aug 01, 2007 1:39 pm
by Smartball
I wrote an instant messaging program in C# once that allowed users to resume previous downloads from other users. The idea actually isn't difficult at all, assuming the files are sent sequentially (starting at byte 0 and ending at the last byte, which I assume it is). When the server is rattling off the server packages to the client, and the client checks to see if it has each package, if it encounters a package that it doesn't have, it simply needs to check if .tmp exists (as Bane said). If it does exist, then all it needs to do is send back the file size of the .tmp file to the server, and the server should begin transferring the file to the client at that offset, and the client should begin appending. If the file doesn't exist, the client just sends back 0 so the server begins sending from the 0th offset, or the beginning of the file. It's easy in theory and was easy to implement as well, as long as the files are sent from server to client starting at byte 0 and ending at the last byte (I can't imagine any reason why it wouldn't be sent in order, but just thought I'd stress it lol.)

Re: Save unfinished cache files

Posted: Thu Aug 02, 2007 2:43 pm
by TCP_Wolf
It __might__ be a good idea to have a "resume window" of 1 or 2k in size, because sometimes when a download breaks the last few bytes are corrupted. Or as a safeguard, the client could simply discard the last couple of bytes when it broke. By principal it does work of course....

Re: Save unfinished cache files

Posted: Mon Jan 18, 2010 12:05 am
by [§Ŕ] ŤhěxĐâŕkśîđěŕ
Bump for justice.

I guess it'd also then keep the download and not restart it from the beginning if the server switches maps, that's really annoying when it happens.

Re: Save unfinished cache files

Posted: Tue Jan 19, 2010 2:00 pm
by Hyper
I had this idea while seeing those temp cache files..
Would it be possible to make it save unfinished downloads in some directory and next time ur about to download the same file again it would continue downloading it. It's just so frustating when ur downloading a 10 MB file at 95 %, then server map change and you have to start over.. or is the idea too time consuming to make?
The idea sounds fine to me. Not a critical feature but surely nice to have. But why don't you make it yourself, now you're in the dev team? ;)

Re: Save unfinished cache files

Posted: Tue Jan 19, 2010 8:35 pm
by .:..:
The idea sounds fine to me. Not a critical feature but surely nice to have. But why don't you make it yourself, now you're in the dev team?  ;)
Actually it's very hard for me to do that without the source code of Unreal Engine 1. Even so it might not be worth the time and all the trouble with debugging it when HTTP redirecting handles downloading very fast anyway.