logo
Main

Forums

Downloads

Unreal-Netiquette

Donate for Oldunreal:
Donate

borderline

Links to our wiki:
Wiki

Walkthrough

Links

Tutorials

Unreal Reference

Usermaps

borderline

Contact us:
Submit News
Page Index Toggle Pages: 1 Send TopicPrint
Normal Topic Experiments in AI Upscaling (Read 807 times)
DaveW
Junior Member
**
Offline



Posts: 70
Joined: Apr 4th, 2004
Experiments in AI Upscaling
Jan 17th, 2020 at 8:29pm
Print Post  
Hello all

I've recently been tinkering with AI Upscaling (again - I released an AI upscale pack for Deus Ex last year) and decided to try it on Unreal textures. Not sure if anyone's done this yet.

So far I think the results are pretty interesting. This is using ESRGAN with the 'Manga109attempt' model, which seems to be best suited to game textures:

https://imgsli.com/MTExMzY

https://imgsli.com/MTExMzc

https://imgsli.com/MTExMzg

https://imgsli.com/MTExMzk

https://imgsli.com/MTExNDA


The results are less dramatic than the URP, which is also the case with New Vision - but it just adds an extra bit of sharpness when playing at high res without changing the aesthetic / style of the original.

Currently exporting the textures is a bit of a pain and I'm not sure how to do the textures in .u files (mutators seem to be the best way? No idea how to do that) but if nothing else I think this is pretty cool!
  
Back to top
 
IP Logged
 
Pitbull™
Forum Administrator
*****
Offline



Posts: 1100
Location: Between Venus & Mars
Joined: Oct 4th, 2002
Gender: Male
Re: Experiments in AI Upscaling
Reply #1 - Jan 18th, 2020 at 2:42am
Print Post  
Looks impressive.
  

Green River

LOADING HATERS..████████████] 99% Complete. 
Back to top
WWWGTalk  
IP Logged
 
Hellkeeper
Global Moderator
*****
Offline


Soulless Automaton

Posts: 2866
Location: France
Joined: May 21st, 2008
Gender: Male
Re: Experiments in AI Upscaling
Reply #2 - Jan 18th, 2020 at 11:36am
Print Post  
Let me get this straight, this is all computer-generated upscaling via an AI? The result is close to prefet and needs no hand work? Very cool.
  

You must construct additional pylons.
Back to top
WWW  
IP Logged
 
DaveW
Junior Member
**
Offline



Posts: 70
Joined: Apr 4th, 2004
Re: Experiments in AI Upscaling
Reply #3 - Jan 18th, 2020 at 12:00pm
Print Post  
Exactly, each texture takes approximately 5 seconds to process on my PC (Ryzen 3700x / RTX 2070), since most of the process time is in actually building the models in the first place. In this case, I'm using someone else's trained model - which I think is the best approach. A model is basically a set of instructions telling the AI how to upscale particular details, trained on a massive dataset of other textures.

It might be possible to train a model on Unreal's textures themselves, using the official high resolution textures that were released with UT, but I think that might give more inconsistent results (being a small sample size). So far, these have been pretty good.

The great thing about this technique is it can also be applied to any custom textures quite easily. I will upload some more examples later.
  
Back to top
 
IP Logged
 
Lightning_Hunter
Senior Member
Betatester
****
Offline



Posts: 275
Location: somewhere on Earth
Joined: Mar 4th, 2003
Gender: Male
Re: Experiments in AI Upscaling
Reply #4 - Jan 19th, 2020 at 5:19am
Print Post  
I don't mean to steal your thunder, but Ahaigh01 did exactly what you are describing with the UnrealHD texture pack 2.0! He first ran all the textures through various AI upscalers, then tweaked the small details by hand:
https://www.moddb.com/games/unreal/addons/unreal-hd-textures
The only textures not upscaled were the very best ones by Diehard for the original Unreal HD pack, but lots of the lower quality ones were redone.

As for the player skins, I personally tackled those based on AI upscaled skins by Ahaigh01, but I almost completely reworked those by hand for my UnrealHD mod. The rest of the game skins I did without upscalers:
https://www.moddb.com/mods/high-resolution-unreal-skins/downloads/unrealhd-v2
  

For High-Res Unreal skins, click here.
Back to top
IP Logged
 
DaveW
Junior Member
**
Offline



Posts: 70
Joined: Apr 4th, 2004
Re: Experiments in AI Upscaling
Reply #5 - Jan 24th, 2020 at 4:55pm
Print Post  
I think if I created a full pack out of this, it would be pure AI upscale and no manual tweaking, to get as close to the original look as possible. At least then there's the option of a 'purist' release, and the existing ones with some manual texture updates.

The problem is artifacting on some textures, but I'm getting a bit better at reducing those now.
  
Back to top
 
IP Logged
 
Lightning_Hunter
Senior Member
Betatester
****
Offline



Posts: 275
Location: somewhere on Earth
Joined: Mar 4th, 2003
Gender: Male
Re: Experiments in AI Upscaling
Reply #6 - Feb 8th, 2020 at 3:51am
Print Post  
DaveW wrote on Jan 24th, 2020 at 4:55pm:
I think if I created a full pack out of this, it would be pure AI upscale and no manual tweaking, to get as close to the original look as possible. At least then there's the option of a 'purist' release, and the existing ones with some manual texture updates.

The problem is artifacting on some textures, but I'm getting a bit better at reducing those now.


I don't wish to de-motivate you, but AI Upscaling has all sorts of problems beyond artifacts that, if not manually reworked, looks worse than the original textures in many cases. Ahaigh01 has been working with upscalers for quite some time now, and I have seen him run thousands of textures through them.  I personally pointed out hundreds of textures that turned out kind of awful looking without manual tweaks.  The biggest example of this is when upscalers attempt to upsize screws, decals, and other sharp details. The upscaler will often guess what these details are, and butcher them in the process.  The result is a decal or screw that no longer looks like a decal or screw. Such details always need manual reworking to look like they are supposed to.  Otherwise, they look terrible and you are better off sticking to the original textures.

At least, that is my two cents, but don't let me stop you from working on textures of course.  We could always use more texture artists around here. 
  

For High-Res Unreal skins, click here.
Back to top
IP Logged
 
yrex .
Betatester
Offline


Oldunreal member

Posts: 217
Joined: May 6th, 2015
Re: Experiments in AI Upscaling
Reply #7 - Feb 8th, 2020 at 4:39pm
Print Post  
I have no idea if this would work and look good, and there's probably no tool that would do that, but if you care about accuracy, you could simply keep the original texture for the lower-res mipmaps.
  

Quality content | mail me instead of PMing: ampoyrex(a)wp(.)pl
Back to top
 
IP Logged
 
Page Index Toggle Pages: 1
Send TopicPrint
Bookmarks: del.icio.us Digg Facebook Google Google+ Linked in reddit StumbleUpon Twitter Yahoo