Save unfinished cache files

[.:..:]

I had this idea while seeing those temp cache files..
Would it be possible to make it save unfinished downloads in some directory and next time ur about to download the same file again it would continue downloading it. It's just so frustating when ur downloading a 10 MB file at 95 %, then server map change and you have to start over.. or is the idea too time consuming to make?

[Shivaxi]

omfg...this is a really good idea...i totally agree .:..: I dunno how this would work but it does seem possible

[Bane]

I'd assume that doing this would be relatively simple. While downloading the file, occasionally dump what has been downloaded so far into the file .tmp. When starting a download, check to see if this file exists. If so, open the file and resume the download from there. I've never tried doing anything with saving files in C++, but I can't imagine that it would be too difficult to implement ths.

[Smirftsch]

didnt mess with these things yet, but i think the problem is maybe more the resume than to save it.

[TCP_Wolf]

Yea sounds like requiring a few intelligent changes to both server and client. File handling itself is easy (relatively) and fast in C++. But you know things are ALWAYS more complicated than they first appear :-O

[DieHard SCWS]

I often notice a Cache0000.tmp (Cache0001.tmp, Cache0002.tmp, etc.) files sitting in the root of my Unreal folder.


To me it always looked like a cache Temp like discussed here, but disfunctional. In other words, it looks like the Temp is created, but is indeed unable to resume. Unless you guys tell me those files are something different, but that is what i always thought........
.
.
.

[Smirftsch]

no, you are right. It is what you think.
But on the other hand, it might be not necessary at all when the download is fixed. I tried it with havoks suggestion to set the tickrate of the server to 70- long time i ago i saw such a fast download. So there must be a way to fix that.

[Bane]

huh. I had no idea that's what those were for. I thought it was saved information from the pre-caching (not that I know what that is...). I just opened one of them in a hex editor and didn't see the GUID for it. If the GUID isn't in there (and it isn't in the file name), there's no way it could possibly resume. I'm guessing it's just a temp file that stores the information as it's received so it doesn't all have to stay in memory at once. The file format for these could probably be modified somewhat to allow recovering downloads.

[.:..:]

Well once I saw some map data in 1 of those temp files like "Brush103" "Nali1" etc...
So, I think it stores the currently finished downloads there while game is downloading, then when its finished it moves the file to cache folder and renames it. Maybe some unexpected shut down makes unreal not to delete them when download cancels (crash, exit?).

[Bane]

Actually, even pressing F10 causes the files to remain. Perhaps they had intended support for incomplete downloads at some point and just never finished it. Even if the files did persist after a crash when they should've been deleted, unreal could just clean them all out on next startup, like with purgecachedays

[Smartball]

I wrote an instant messaging program in C# once that allowed users to resume previous downloads from other users. The idea actually isn't difficult at all, assuming the files are sent sequentially (starting at byte 0 and ending at the last byte, which I assume it is). When the server is rattling off the server packages to the client, and the client checks to see if it has each package, if it encounters a package that it doesn't have, it simply needs to check if .tmp exists (as Bane said). If it does exist, then all it needs to do is send back the file size of the .tmp file to the server, and the server should begin transferring the file to the client at that offset, and the client should begin appending. If the file doesn't exist, the client just sends back 0 so the server begins sending from the 0th offset, or the beginning of the file. It's easy in theory and was easy to implement as well, as long as the files are sent from server to client starting at byte 0 and ending at the last byte (I can't imagine any reason why it wouldn't be sent in order, but just thought I'd stress it lol.)

[TCP_Wolf]

It __might__ be a good idea to have a "resume window" of 1 or 2k in size, because sometimes when a download breaks the last few bytes are corrupted. Or as a safeguard, the client could simply discard the last couple of bytes when it broke. By principal it does work of course....

[[§Ŕ] ŤhěxĐâŕkśîđěŕ]

Bump for justice.

I guess it'd also then keep the download and not restart it from the beginning if the server switches maps, that's really annoying when it happens.

[Hyper]

I had this idea while seeing those temp cache files..
Would it be possible to make it save unfinished downloads in some directory and next time ur about to download the same file again it would continue downloading it. It's just so frustating when ur downloading a 10 MB file at 95 %, then server map change and you have to start over.. or is the idea too time consuming to make?

The idea sounds fine to me. Not a critical feature but surely nice to have. But why don't you make it yourself, now you're in the dev team?

[.:..:]

The idea sounds fine to me. Not a critical feature but surely nice to have. But why don't you make it yourself, now you're in the dev team?  ;)

Actually it's very hard for me to do that without the source code of Unreal Engine 1. Even so it might not be worth the time and all the trouble with debugging it when HTTP redirecting handles downloading very fast anyway.