Help with an experiment

[BOZO]

Can one of you STC3 wizards help me out with an experiment?

I would like you to create an STC3 High Res version of a mesh texture and I will see if I can figure out a way to be able to replace it successfully and retain UV coordinate information on meshes without requiring a full mesh reimport.

Lets start with a simple texture - the one used for the Asbestos Suit - UnrealI.AAsbSuit1

Thanks!!

[Pitbull]

Can one of you STC3 wizards help me out with an experiment?

I would like you to create an STC3 High Res version of a mesh texture and I will see if I can figure out a way to be able to replace it successfully and retain UV coordinate information on meshes without requiring a full mesh reimport.

Lets start with a simple texture - the one used for the Asbestos Suit -  UnrealI.AAsbSuit1

Thanks!!

Boz you always think of the cool stuff.

[Xavious]

I would very strongly recommend you start with something even simpler: the steelbox...

Or even the cube lol (or the dice, as thats what it looks like)

[BOZO]

Actually, thats not such a bad idea to work on the steel box or crate first. So forget making me a nice texture - I'll just use an existing SCT3 texture and try that instead.

[Turboman.]

afaik uvw data isnt limited to the original texture size, meaning you can create a new 1024x1024 (or 8192x8192 for that matter) and still fit the model properly.

although there is a slight LOD problem with models in unreal, afaik the game processes lod pretty heavy meaning texture detail is always reduced into lower detail which isnt a good thing.

[BOZO]

I have been experimenting with how to go about replacing the textures without having to reimport the meshes. The problem is that most of the mesh textures are embedded within the UnrealShare.u and UnrealI.u files and it would be problematic to create a S3TC version of these core files.

A test I did:

I reimported the texture used for the Kevlar suit into the UnrealShare.u package at a higher resolution and it showed up on the model itself just fine. Of course I would not want to do this normally since we do not want to modify the UnrealShare.u or UnrealI.u files themselves. (I discarded my changes) This shows that using higher resolution textures is relatively easy to do though.


You are quite correct in that it does not matter what size the texture is that you import (As long as it is a multiple of the original) and it seems that it will be displayed correctly. The coordinates seem to be stored in a decimal format (0.0 - 1.0) and thus will scale correctly if the new texture is the same shape as the original.


My current thoughts on how to approach this:

Write a mutator class that will dynamically replace textures by using the MultiSkin functionality. This is a built in functionality and should be an easy way to implement many of the dynamic replacements. I was thinking of creating a configuration file that lists classnames and the new replacement texture names and walking through the list to decide when to implement changes.

Then we could create two texture files (with the same GUID and filename) one containing the base PCX textures (same textures as those in the UnrealShare.u and UnrealI.u files) and the second containing the S3TC versions of those textures. The names used for the textures must be exactly the same for each package.

The first file (BASE) will be placed on the servers and will be listed in the ServerPackages list - this one will have the PCX versions of the texture files. New joiners can download this file at runtime and not have any problems.

The second file (S3TC) will be for S3TC capable clients only and will be installed on their computers only.

What we can do is start out with both files being exactly the same at first each containing a copy of all PCX versions of the textures.

As we complete S3TC textures we load them into the 2nd file and overwrite the PCX versions. We then update the mutator configuration file (and possibly the code if needed) also and release a new version for installation by the masses. The configuration file (and mutator code changes) will need to be loaded on the server and the S3TC texture file will need to be downloaded by the users (outside of the server download) and be installed manually.

This will allow smooth interoperability without replacing the base classes. S3TC clients will see high rez textures - standard clients will see standard textures. The only configuring required will be the addition of a mutator call to your startup commands on the server side.

If this seems like it may be a valid workaround approach to the rest of you - I will work on creating a full test case here shortly.

Current drawbacks/details yet to be worked out:

You have to follow the original texture layout exactly (UV mapping) for this to work right - if you try to shift things around on the texture then you have to reimport the mesh with new texture coordinate mapping and this forces you to create a whole new class...and it gets too involved from there on.

Multitexture meshes are a bit more complex to have dynamic texture replacements work correctly. I think I did this successfully in the past but I remember there were some issues involved with it that I don't quite recall at the moment. Single texture meshes should be easy to do though.

We need to add all of these textures from (UnrealShare.u and UnrealI.u) to the S3TC replacement project - and this makes the workload huge on their end. I'm not a texture guy but I will work out the code end of things for this to operate right. Thus we probably need a few more volunteer texture artists.

[BOZO]

Console Commands:

MLOD X

A strength scaling factor centered on 1.0, to scale the strength of the LOD culling- higher values stengthen the LOD fall-off with distance.


MLMODE X

0 Draw everything at full (or at MLFIX-ed) detail.
1 Normal level-of-detail mode.
2 Disable smooth morphing.
3 Ignore the field-of-view effect on LOD (great for debugging!)
4 Disable smooth morphing, ignore field-of-view.


MLFIX X

A float in the range of 0.0-1.0 which, when in MLMODE 0, causes all meshes to be drawn using this proportion of their vertices.


Enjoy!

[Smirftsch]

bozo, i think i can help with that. I got the NDA, signed it and send it today to epic, means if possible to rework the UT code i get to work with unreal again, i start soon working on 227, which will allow me to redo those internal .u files and want to make a s3tc version of it as well. Since Unreal doesnt support merged "normal"/s3tc texture packages like UT does (maybe if i'm allowed to enhance unreal with it, it isnt a problem anymore) i'd maybe forced to make 2 packages, a 227 patch for non and one for with s3tc textures means about that ->

"Write a mutator class that will dynamically replace textures by using the MultiSkin functionality. This is a built in functionality and should be an easy way to implement many of the dynamic replacements. I was thinking of creating a configuration file that lists classnames and the new replacement texture names and walking through the list to decide when to implement changes.

Then we could create two texture files (with the same GUID and filename) one containing the base PCX textures (same textures as those in the UnrealShare.u and UnrealI.u files) and the second containing the S3TC versions of those textures. The names used for the textures must be exactly the same for each package.
"



I'm not used to make grafix and i'm not very good in it, i exactly need the packges which are created here, so making it would be your thing, while implementing will be my sorrow, although i'm pretty sure i need the help of everyone in here.




"You are quite correct in that it does not matter what size the texture is that you import (As long as it is a multiple of the original) and it seems that it will be displayed correctly. The coordinates seem to be stored in a decimal format (0.0 - 1.0) and thus will scale correctly if the new texture is the same shape as the original."

So size doesn't matter ?

tell me your opinion about this all

[BOZO]

I think the best approach will be to start with the workaround I have outlined as a temporary solution while we are updating/correcting the S3TC textures. I expect this will take some time to complete and will include many contributors. Having the textures in an external package (outside of gameplay) will save us greatly from version mismatch problems while we are working on replacing the full texture list. This will also facilitate incremental releases that will not have version mismatch problems we would see with a modification of the core files. Updates on my end would consist mainly of updating an *.ini file and importing textures and releasing the new S3TC texture package for each build release. I forsee little need for code updates once I get the core classes written and operational. It should be easy on the server folks too - all they need to do is overwrite the *.ini file to support any new textures we have implemented. It will not matter if clients have the latest version of the texture package. I don't forsee any issues with having multiple client versions operating at once at all.

Once all of that development work is finished, then we should be able to create a new S3TC version of the UnrealShare.u and UnrealI.u and UPak.u files (I assume we will do it for Gold also) containing all the new textures plus all of the code updates we may have implemented in the meantime and be able to create just one new update patch file for S3TC and a second with the standard textures and any updated code for regular users. The mutator (and workaround) will then become extraneous and will no longer be needed.

Of course if you can implement some means of applying different textures without requiring two packages so much the better.

As far as texture size goes, I'd probably keep them a maximum size of a multiple of 4 of the original size. This should give folks plenty of detail to work with.

Great news about the NDA!! Where do I sign?

[DieHard SCWS]

Gotta run to work, so this is just an fast responce.

Not sure if i can redo those textures now technologiewize.


But for now it would greatly be appreciated if someone would extract all those textures as pcx files for the database. So there is a place to work from.

As far as texture size goes, I'd probably keep them a maximum size of a multiple of 4 of the original size. This should give folks plenty of detail to work with.

All textures ALWAYS needs 2048 x2048 x 2048 as a minumum to be able to add enough detail to begin with, you can always scale them down if you want....


But for now we need all textures (as they are) to be able to study them. If you sent them don't alter them in any way, keep them pcx as the size they were extracted and zip them all in.
.
.
.

[BOZO]

Bringing up an old topic a bit but this covers pretty much the mutator deal we were talking about.

I was also messing around the other day with extracting the nicer textures from the Unreal Beta. I can't seem to use any regular external extractor or export function to get these out easily. I can do screen captures and crop afterwords accordingly but this is time consuming and I'm not certain how well it works overall. Does anyone have an easier way of extracting these textures? These could easily be added to the ver 227 patch so we get a little additional baseline detail for folks not running S3TC without alot of pain.

[ahaigh01]

Diehard wrote:

But for now it would greatly be appreciated if someone would extract all those textures as pcx files for the database. So there is a place to work from.

I don't know if anyone ever did this, but here is a zip file with all the embedded textures from UnrealI.u, UnrealShare.u, and Upak.u.
http://www.streamload.com/kkorker/unreal/U1files/bmp_textures_from_U1pkgs.zip

They are in bmp for now.  If you really need them in pcx let me know, but wotgrealexporter had a problem with pcx

[ahaigh01]

BOZO wrote:

I was also messing around the other day with extracting the nicer textures from the Unreal Beta. I can't seem to use any regular external extractor or export function to get these out easily. I can do screen captures and crop afterwords accordingly but this is time consuming and I'm not certain how well it works overall. Does anyone have an easier way of extracting these textures? These could easily be added to the ver 227 patch so we get a little additional baseline detail for folks not running S3TC without alot of pain.

I tried wotgrealexpoter and utpt on them and could get them to work either.  However if you use the version of ued that comes with the beta you can export them to pcx 1 at a time.  You have to name them manually, but it is a lot easier than screen capturing them.

[Hyper]

I'm a bit worried about the idea of using a mutator to replace client-side textures. I know that client-side mutators will not work when you play online. This is a major problem because many players play Unreal only or mainly as multi-player game.

[TCP_Wolf]

If the mutator changes the "multiskin" attributes serverside, they should replicate to the client. From what I understood that's the reason the client has to download (at least) the normal texture anyway if it has neither the low or high-res textures. Otherwise, what you supposed would be a client mutator which would only work with a client side texture package which is not present at the server... and no that probably wouldn't work for several reasons...

Still, even with the multiskin feature, this will mean trouble I think, because tools that check package sizes and compare server/client values may find discrepancies, even if you adjust the signature... but we will have to see...

[Hyper]

If it depends on a server then it's not useful at all in my opinion. S3TC textures should be purely a client-side thing so every player can decide if he/she wants to use S3TC or not, independend of the taste of the server owner.

[ahaigh01]

I don't know too much about server side stuff, but I hope BOZO will still make it so we can play with it in single player at least.

[DieHard SCWS]

I don't know if anyone ever did this, but here is a zip file with all the embedded textures from UnrealI.u, UnrealShare.u, and Upak.u.  
http://www.streamload.com/kkorker/unreal/U1files/bmp_textures_from_U1pkgs.zip

They are in bmp for now.  If you really need them in pcx let me know, but wotgrealexporter had a problem with pcx

No that is just fine, i said pcx because that is the export supported by Unreal, BMP is just fine





I agree with Hyper completelly, i am not fond on any serversided dependency, the major advantage is that its clientsided !


But it needs to be said, for Single players it still would be cool, and it still would mean the textures will be ready at the time theres a non serversided solution. So the work still could proceed.





As far as bot protection problems.

I think that is the least of our worries, than those simply have to go lol, theres not too many botters left, if any. I think the project has more use than botprotectors.

But in the case of botprotection, i may have an actual complete stupid idea to have both working. And its a simple solution, though i am not gonna share this in public.





The clientsided advantage actually is awsome, the S3TC textures i am doing for the SGC map will also be available for UT. However............. The tool to create a mixed normal/S3TC package for UT is there. so no prob.


But it would mean you have to upload some 300/500 Mb big packages to the server. No need to tell that will be an impossible download. So for UT too, this needs to be done clientsided only. Normal and S3TC should be seperate so theres no hastle.






But as said above, its still a good idea to work on it, at least for Single players and servers willing to support it, but only to the extend that any player S3TC or not always still can join the server normally without any problems.
.
.
.

[JediMa_TNF]

Can I have some faq about them? Where do I get em? and how can i install em? ty

[DieHard SCWS]

Its a bit clouded question, i just asume you mean the S3TC textures, Here is the link to the installation guides and downloads:


http://www.celticwarriors.net/website/S3TC/S3TCInstallation/S3TCInstallation.htm


(installation guide for UT is still unfinished though)

.
.
.

[BOZO]

Ok folks,

I got the mutator working using the multiskin approach methodology.

This requires the two texture packages involved have the same name and GUID to operate correctly.

The first package will contain textures using standard resolution. This will be placed on the server and listed in the serverpackages.

The second package will start out being exactly the same as the first package and as we make the higher resolution textures we will add and replace textures accordingly. The client will load this package. This will be the one that is updated frequently.

If someone joins without having either package loaded then they will download the standard resolution package.

The server runs the mutator and it auto replaces textures at runtime. Each client will see a different texture depending on which package they have loaded.

I have written this so that all I need to do to add support for a new updated texture is to update the *.ini file that is loaded on the server. A new *.u package will not be required for each release unless I decide to add some new functionality in the future.

This methodology is operating very similar to how the S3TC support has been implemented.

Things needed:

All textures from UnrealShare extracted and added to a seperate texture file. This will be the base texture file to be used on the server. Try to keep things the same in the new package.

Example: Add the texture used by the KevlarSuit as UnrealShare.Skins.AkevSuit1
to the new package. This will mirror the original and make things easier on everyone when maintaining the *.ini file.


A few higher resolution textures to add to a test package so I can release a beta version for folks to test out. What we may want to do is take the extracted textures from the Unreal Beta and add these to the package to get a head start on things.


Lets work on textures from UnrealShare only at the moment please. Someone else please volunteer to do these things - my time is very limited at the moment.

[DieHard SCWS]

I have been working on this for 5 hours last night when i saw this post. I concluded that the package can be done, though it needs to be said, its a very strange package. I do not completelly understand how the package can be assembled like that to begin with.


There is a limit in the package, we cannot handle the Wet animations, or other animations  like that.


For now, i would like to know which specific texture(s) you want to try, so i can make dymmies for those textures. Those textures should be textures that easelly can be spotted on various maps.


And by dummies i mean, complete nonsence, but high resolution 24 bit textures, that is unless Alhaig can provide some high resolution results that are present in the package mentioned.


In the last case i need the textures in full rendered resolution while having the exact name (including capitals and non capitals) as either TIFF or PSD file (preferably PSD).





As i said, i did alot of experimenting on the package last night, i also found out that i can safe the UnrealShare package as utx file.


The interesting thing that occured was, i renamed it to UnrealShare.u and added it back to my game. And incredibly enough my game worked just fine. I tried to go online, but there it stopped due to a version missmatch. Than i tried to see if i could rename the original file simply to an .utx file and test if the Mergertool was able to handle it, while trying to merge some DDS textures to it.


This however chrashed the Mergertool, but i am still curious, if now Kerilk could have a go after all to make the tool compatible with this specific file type.



But as far as your test BOZO, i just need to know which textures you want to have for testing only.
.
.
.

[DieHard SCWS]

If it depends on a server then it's not useful at all in my opinion. S3TC textures should be purely a client-side thing so every player can decide if he/she wants to use S3TC or not, independend of the taste of the server owner.

As can be read above already, i agreed on this completelly with Hyper.........


However


Since the serverside aproach is done with a unique .utx file, in this case UnrealShare.utx, it will not bite anyone or anything. In case Kerilk finds a way to have the Mergertol compatible with UnrealShare.u than this can be done also.


So for now theres a few beneficial parts in this:

- for the time being, the textures at least are being created (Alhaigh)
- for the time being, if this all works, than you CAN play with the textures on all servers that DO use the upgrades.
- for the time being, if it works, than theres no mismatches on the servers, since the system still works the same, if i update the S3TC package, and the server does not follow in time, than the updates simply will not be visible, like you already are used with the current textures..............


Downside:

Prob only one as far as i can see.
- Its unclear to me how the mutator will handle, or can handle in SinglePlayer.
.
.
.

[ahaigh01]

Excellent news!

Yes I can supply some of the textures right now.  The plant and tree skins are in fact found in unrealshare.u.  There are 17 plant and tree skins and I have completed 13 of them with the exception of adding alpha channels the ones that use masking.  For my renders in 3dsmax I have been creating a separate ‘opacity map’ which does the same thing, but I haven’t yet tried creating a final s3tc texture yet.  Diehard will be able to do that with ease  I am beginning to upload some zip files right now to my file storage site.  I will PM Diehard and BOZO as to where to pick the files up.  They will contain:

-PSD file, 2048x2048, resolution = 2048 – not flattened – no alpha channel yet
-PCX original texture
-txt file with any important info

---------------------

I’ve also poked around with the original textures from unrealshare.u.  I am working with wotgreal exporter and utpt beta 5.  I have found that both programs export corrupt PCX files but work fine for BMP.  I exported all the textures from unrealshare.u to BMP using wotgrealexporter, and then batch converted them back to PCX with Irfanview.  There are 595 textures in all.  I uploaded a zip file of them to my storage site in PCX format – that can be the master file for the originals if needed.

As i said, i did alot of experimenting on the package last night, i also found out that i can safe the UnrealShare package as utx file.

For creating the utx package with the original textures - Yes, you can rename unrealshare.u to .utx, if thats what you mean.  That is cool because then all the origial PCX textures are already in there the way they are supposed to be.  But the downside is that there is also lots of other stuff in there too, so it will be about 3 or 4 times bigger than it would need to be.  Question for BOZO- is it possible to delete unwanted things out of unrealshare.u and rename it to a .utx package?

Or would you build the package with the Make command?

Or would you make the utx package by adding the textures 1 by 1 in ued?

In the last case, this is my understanding of how the unrealshare.utx (assuming that is what the utx file will be named) could be built:

1.  In ued1 -> textures -> import, start a new .utx package called unrealshare.utx
2.  For each texture imported, fill in the name of the Group that it came from in the original unrealshare.u. UTPT shows what group the texture is from.
3.  Set the checkbox for Masked if appropriate for that texture.
4.  Leave Generate Mipmaps checked (I’m guessing – is this right for mesh skins?)
5.  Save

[BOZO]

Ok. Sounds like we are far enough along to produce a test case.

I think what we need to do first is start assembling the base file that has all of the extracted static (non procedural) textures from UnrealShare.u in it. What I was thinking of doing was creating a dummy class that imports all of these files with the correct settings and naming convention from a script perspective using the -make command. I will write this end of it and assemble this package if you folks will get me all of the textures collected together in the *.pcx format. (I have to look up the information on these anyways - naming conventions - etc - to determine how to replace textures correctly in the meshes so it doesn't represent too much more work for me) I think it will work out best if we reimport the textures rather than try to modify the file with all the other stuff loaded.

Lets also start collecting all of the S3TC textures so when the above package is completed we can create our first test package. DieHard and ahaig you guys get to work this end of things.

For the test package lets start with the Trees and Plant textures that ahaig has put together so far and add in the texture for Kevlar Suit, the Dispersion Pistol, and the Manta. This should give us a good first test case.

Issue #1 to resolve:
As an alternative to naming our new package UnrealShare.utx and moving our texture search priority for the Paths variable in the Unreal.ini file to the highest priority: Example: Paths=..\Textures\*.utx
Paths=..\System\*.u
Paths=..\Maps\*.unr
We could name the file something like UnrealShare_HR.utx just to make sure nobody gets it confused with the standard UnrealShare.u package. It should work just fine this way and people will not have to mess with the textures priority. I'm not certain if that will cause problems down the road. Anyone see something I not thinking about with this?

Issue #2 to resolve:
I think all that Kerilk's tool does is add a reference to DrawScale and Format to each new texture being replaced and keeps the GUID the same for each package. Thus we should be able to use this for all of the S3TC textures we are planning to replace without any problems cropping up.
As stated before the procedural textures are problematic to convert. I have been able to get them upgraded to a higher resolution operation using a hex editor but there seems to be a hard coded limit on the size they can be - no greater than 1024x1024. I think I will continue working on coming up with some sort of workaround to get these operating at a higher resolution. I may see about extracting these from the UnrealShare package and placing them all in a seperate package and working on it from there.

Issue #3 to resolve:
I will see about creating an easy way to run this mutator for Single Player operation. I think it is just a matter of figuring out a good way to get the mutator to autoload. It should work just fine once loaded.

Issue #4 to resolve:
I would like to see about maintaining the same Palette usage as the textures in UnrealShare.u to minimize memory overhead usage between the packages. I think I can specify some of this during import. I will have to review all of this to verify for sure.