For direct access use https://forums.oldunreal.com
It's been quite a while since oldunreal had an overhaul, but we are moving to another server which require some updates and changes. The biggest change is the migration of our old reliable YaBB forum to phpBB. This system expects you to login with your username and old password known from YaBB.
If you experience any problems there is also the usual "password forgotten" function. Don't forget to clear your browser cache!
If you have any further concerns feel free to contact me: Smirftsch@oldunreal.com

Unfinished Language features.

Here you can add and find information and help for other Unreal Engine titles. Currently for UE1 and UE2 based software.
Post Reply
User avatar
han
Global Moderator
Posts: 686
Joined: Wed Dec 10, 2014 12:38 am

Unfinished Language features.

Post by han »

UnrealScript language features like an FArray/TMap interface is sth. which seems to be at least supported by some degree by the script compiler, so I tinkering with the idea of trying to experimenting with runtime support for this (e.g. add needed native functions, which are hooked in to some degree).

However as I've never touched this before and actually doesn't have a list of these features I'm asking you guys if you can name the things you know of. This doesn't have to be DeusEx specific, but rather for any unreal 1 game, so i have an idea what i can try and what probably fails.
HX on Mod DB. Revision on Steam. Löffels on Patreon.
User avatar
.:..:
OldUnreal Member
Posts: 1634
Joined: Tue Aug 16, 2005 4:35 am

Re: Unfinished Language features.

Post by .:..: »

FArray is base class of TArray.
TArray is dynamic array object in C++ codes, as for TMap, it is hash mapping for paired values, for example:
TMap, can be used to quick find a matching class reference with a string.

Dynamic array support is broken in UnrealScript on UnrealEngine 1.
Map is fully unsupported in UnrealScript on all UnrealEngine versions.
1823223D2A33224B0 wrote:...and now im stuck trying to fix everything you broke for the next 227 release xD :P
(ಠ_ಠ)
User avatar
han
Global Moderator
Posts: 686
Joined: Wed Dec 10, 2014 12:38 am

Re: Unfinished Language features.

Post by han »

Actually i wanted to type TArray, not FArray.
I know that TArray support is broken, but i just saw TArrayNoInit in Brother Bear, and actually DeusEx has sth like this in it's uc source, but i seems to be just used to have a struct of the right size in place:

Code: Select all

// ----------------------------------------------------------------------
// UnrealScript version of TArray/FArray (originally in Object.uc; moved
// here after removal in version 224)
struct DynamicArray
{
      var const int Num, Max, Ptr;
};
var const native DynamicArray displayBuffer;
And as BrotherBear uses a different syntax anyway e.g.

Code: Select all

var array AuxAnims;
However at some time i read sth. about broken script compiler features refering TArray and iirc even TMap stuff, that the script compiler at least notices a bit of it's syntax and stuff. However, the question i want to investigate is if the compiler actually compiles some of that TArray related code and if he does, did he create opcodes for it? If it does that i could try adding a function to GNatives implementing that functionality. Iirc i saw that Brother Bear had two hardcoded names, added exec's or sth. (can't find it again right now) comparted to ut432 which sounded like TArray support, but actually it did had the first of a set of three functions which sounded like sth which is used to access an TArray. So I'm hoping that the compiler actually creates an opcode for it, hence I'm asking for detailed info about what is already known, so I don't have to start from scratch.
Last edited by han on Mon Jan 26, 2015 4:06 pm, edited 1 time in total.
HX on Mod DB. Revision on Steam. Löffels on Patreon.
User avatar
.:..:
OldUnreal Member
Posts: 1634
Joined: Tue Aug 16, 2005 4:35 am

Re: Unfinished Language features.

Post by .:..: »

Well the pre-224 version with "DynamicArray" struct has absolutely no UnrealScript support for dynamic arrays.

224 and later UScript compiler does produce compiled code for array entry accessor, but the native function itself is unimplemented, so if you want you CAN actually override that GNative entry to a fixed version of the accessor:

Code: Select all

GNatives[EX_DynArrayElement] = &MyCustomObject::execCustomArElement;
That way this will be functional:

Code: Select all

var array A;
A[0] = 25;
But you will still need to implement your own native function to modify array size.
1823223D2A33224B0 wrote:...and now im stuck trying to fix everything you broke for the next 227 release xD :P
(ಠ_ಠ)
User avatar
han
Global Moderator
Posts: 686
Joined: Wed Dec 10, 2014 12:38 am

Re: Unfinished Language features.

Post by han »

Thanks!
Does the static array accessor do proper bounds check? Probably not.. as it will try to access UFixedArrayProperty::Count, which is inside the alignment/padding following the UArrayProperty. So i guess i should start with trying to implement a save version of it, which behaves like the static accessor if out of bounds.
HX on Mod DB. Revision on Steam. Löffels on Patreon.
User avatar
.:..:
OldUnreal Member
Posts: 1634
Joined: Tue Aug 16, 2005 4:35 am

Re: Unfinished Language features.

Post by .:..: »

You can rather easily find UE source codes with google, from there you can see how it's implemented.
1823223D2A33224B0 wrote:...and now im stuck trying to fix everything you broke for the next 227 release xD :P
(ಠ_ಠ)
User avatar
Higor
OldUnreal Member
Posts: 51
Joined: Tue Oct 21, 2014 3:19 pm

Re: Unfinished Language features.

Post by Higor »

Given i'm not a licensee or anything, am I allowed to post what I used in XC_GameEngine to add dynamic array support for UT?
Even if I made unrealscript code compiled with it cross-compatible with 227?
User avatar
[]KAOS[]Casey
OldUnreal Member
Posts: 4497
Joined: Sun Aug 07, 2011 4:22 am
Location: over there

Re: Unfinished Language features.

Post by []KAOS[]Casey »

As far as I'm concerned, if it's doable in public headers using your own code, it's fine.

Unrealscript for U1 probably generates the same opcodes for generating dynamic arrays as UT. I dont think the compiler for 227 was changed to support dynamic arrays further, only native functions to access TArray functions.
User avatar
Smirftsch
Administrator
Posts: 8999
Joined: Wed Apr 29, 1998 10:00 pm
Location: NaPali
Contact:

Re: Unfinished Language features.

Post by Smirftsch »

as for 227, well, we have some additions, dots really did a good job there... :)
Dynamic Array support has been improved:

Added dynamic array length accessing functions: native(640) static final function int GetArraySize( ArrayProperty ArProp ); native(641) static final function bool InsertArrayIdx( ArrayProperty ArProp, int Offset, optional int Count ); native(642) static final function bool RemoveArrayIdx( ArrayProperty ArProp, int Offset, optional int Count );

static native(238) final function string Locs( string InStr ); native(239) static final function string ReplaceStr( string Text, string FindStr, string ReplaceWith, optional bool bCaseInsensitive ); native(240) static final function bool SortArray( ArrayProperty Prop, Function SortCode ); native(241) static final function bool SortStaticArray( Property Prop, Function SortCode, optional int SortSize ); Which does convert string to lower case, replace part of some string, sort dynamic and static arrays (in a very performance effective way).

Added new defaultproperties macros for dynamic arrays: Array.Add(Value), Array.Remove(Value), Array.Empty
Given i'm not a licensee or anything, am I allowed to post what I used in XC_GameEngine to add dynamic array support for UT?
Even if I made unrealscript code compiled with it cross-compatible with 227?
as long its based on public headers as Casey already said, everything is fine.
Last edited by Smirftsch on Wed Mar 04, 2015 11:32 am, edited 1 time in total.
Sometimes you have to lose a fight to win the war.
User avatar
han
Global Moderator
Posts: 686
Joined: Wed Dec 10, 2014 12:38 am

Re: Unfinished Language features.

Post by han »

Not directly an unfinished feature, but i wrote a simple hack for script include support:

Code: Select all

// Stripped of everything which would need other depencencies.
struct FFeedbackContextAnsi : public FFeedbackContext
{
      // Variables.
      INT SlowTaskCount;
      INT WarningCount;
      FContextSupplier* Context;
      FOutputDevice* AuxOut;
};
extern DLL_IMPORT class WLog* GLogWindow;

UBOOL URevisionEditorEngine::SafeExec( const TCHAR* Cmd, FOutputDevice& Ar )
{
      const TCHAR* Str = Cmd;

      // Hack for #exec include.
      if( ParseCommand(&Str,TEXT("INCLUDE")) )
      {
            if ( GLogWindow )
            {
                  Ar.Logf( NAME_ExecWarning, TEXT("Include command not supported inside UnrealEd.") );
                  return 1;
            }
            FString File, Text;
            if ( !Parse(Str,TEXT("FILE="),File) )
            {
                  Ar.Logf( NAME_ExecWarning, TEXT("No include file specified."), *File );
                  return 1;
            }
            if( !appLoadFileToString( Text, *File ) )
            {
                  Ar.Logf( NAME_ExecWarning, TEXT("Include file %s not found."), *File );
                  return 1;
            }

            // TODO: Find another way to aquire ScriptCompiler.
            //       Right now assumption is made that if GLogWindow==NULL
            //       GWarn is an FeedbackContextAnsi. MEGA-BARF.
            FFeedbackContextAnsi* AnsiWarn  = (FFeedbackContextAnsi*)GWarn;
            FScriptCompiler* ScriptCompiler = (FScriptCompiler*)AnsiWarn->Context;
            UClass* Class = ScriptCompiler->Class;

            // Split Script text in part before #exec include and after.
            FString Left, Right;
            INT Start;
            Left  = Class->ScriptText->Text;
            Left  = Left.Left( ScriptCompiler->InputPos );
            Start = Max(Max(Left.InStr(TEXT("\r"),1),Left.InStr(TEXT("\n"),1))+1,0);
            Left  = Left.Left( Start );
            Right = FString( ScriptCompiler->Input+ScriptCompiler->InputPos );

            // Hook in changed ScriptText.
            Class->ScriptText->Text  = Left+TEXT("\r\n")+Text+Right;
            ScriptCompiler->Input    = *Class->ScriptText->Text;
            ScriptCompiler->InputLen = Class->ScriptText->Text.Len();
            ScriptCompiler->InputPos = Start;
            return 1;
      }
      else
            return Super::SafeExec( Cmd, Ar );
}
So now one can do sth. like this:

Code: Select all

#exec include file="Classes\Test.inc"
Last edited by han on Wed Mar 11, 2015 6:04 pm, edited 1 time in total.
HX on Mod DB. Revision on Steam. Löffels on Patreon.
User avatar
han
Global Moderator
Posts: 686
Joined: Wed Dec 10, 2014 12:38 am

Re: Unfinished Language features.

Post by han »

I just had a wicked idea how to hack in features to the script compiler. E.g. if you compile a map declaration the uc compiler throws an exception. So basic idea is to attach some kind of debugger to the process, use it to run own code where it had thrown before and afterwards resume at the same position the exception was thrown. Sounds pretty dirty, but might be worth to investigate..
HX on Mod DB. Revision on Steam. Löffels on Patreon.
User avatar
han
Global Moderator
Posts: 686
Joined: Wed Dec 10, 2014 12:38 am

Re: Unfinished Language features.

Post by han »

So I finally dived deeper into adding dynamic array support for Deus Ex.

Getting P_GET macros for FArray/TArray is straight forward, as FString is more
less just a fancy TArray, so one can just go from there.

Code: Select all

#define P_GET_FARRAY(var)             FArray var;                          Stack.Step( Stack.Object, &var    );
#define P_GET_FARRAY_OPTX(var,def)    FArray var(def);                     Stack.Step( Stack.Object, &var    );
#define P_GET_FARRAY_REF(var)         FArray var##T; GPropAddr=0;          Stack.Step( Stack.Object, &var##T ); FArray* var = GPropAddr ? (FArray*)GPropAddr:&var##T;
#define P_GET_TARRAY(typ,var)         TArray var;                     Stack.Step( Stack.Object, &var    );
#define P_GET_TARRAY_OPTX(typ,var,def)TArray var(def);                Stack.Step( Stack.Object, &var    );
#define P_GET_TARRAY_REF(typ,var)     TArray var##T; GPropAddr=0;     Stack.Step( Stack.Object, &var##T ); TArray* var = GPropAddr ? (TArray*)GPropAddr:&var##T;
Note that when using Objects with the P_GET_TARRAY macro you want to the pointer type for typ.

We can now go ahead and use P_GET_ARRAY_REF macro to implement a native function to retrieve the array count for any type.

Code: Select all

native(3100) static final function int DynIntArrayCount( Array Array );

void UHXObject::execDynIntArrayCount( FFrame& Stack, RESULT_DECL )
{
      guard(UHXObject::execDynArrayCount); // Missing "Int" is intentional.
      P_GET_TARRAY_REF(INT,Array)
      P_FINISH;

      *(INT*)Result = Array->Num();
      unguardexec;
}
Note that the P_GET_TARRAY_REF may not the best implementation, as one can potentially avoid creating the extra fallback copy, but I'm not yet comfortable enough with the script execution system to just head for this. :)

You can do macros to speed up the process. Macros in UnrealScript would probably be also helpful here. ^^'

Code: Select all

#define IMPLEMENT_DYNARRAYCOUNT(cls,type) \
      void cls::execDyn##type##ArrayCount( FFrame& Stack, RESULT_DECL ) \
      { \
            ((UHXObject*)this)->UHXObject::execDynIntArrayCount( Stack, Result ); \
      }

IMPLEMENT_DYNARRAYCOUNT(UHXObject,Byte);
IMPLEMENT_DYNARRAYCOUNT(UHXObject,Float);
IMPLEMENT_DYNARRAYCOUNT(UHXObject,String);
IMPLEMENT_DYNARRAYCOUNT(UHXObject,Name);
IMPLEMENT_DYNARRAYCOUNT(UHXObject,Object);
IMPLEMENT_DYNARRAYCOUNT(UHXObject,Guid);
IMPLEMENT_DYNARRAYCOUNT(UHXObject,Vector);
IMPLEMENT_DYNARRAYCOUNT(UHXObject,Plane);
IMPLEMENT_DYNARRAYCOUNT(UHXObject,Rotator);
IMPLEMENT_DYNARRAYCOUNT(UHXObject,Coords);
IMPLEMENT_DYNARRAYCOUNT(UHXObject,Scale);
IMPLEMENT_DYNARRAYCOUNT(UHXObject,Color);
IMPLEMENT_DYNARRAYCOUNT(UHXObject,BoundingBox);
IMPLEMENT_DYNARRAYCOUNT(UHXObject,BoundingVolume);
However, a HUGE downside is that the native function needs to be per type, however we can at least get away with just having to supply DynObjectArrayCount as it also works for any subclass, and well the next best way is to declare it for any other type we may want to use. Including structs also declared in Core.Object (Deus Ex):

Code: Select all

native(3100) static final function int DynIntArrayCount( Array Array );
native(3101) static final function int DynByteArrayCount( Array Array );
native(3102) static final function int DynFloatArrayCount( Array Array );
native(3103) static final function int DynStringArrayCount( Array Array );
native(3104) static final function int DynNameArrayCount( Array Array );
native(3105) static final function int DynObjectArrayCount( Array Array );
native(3106) static final function int DynGuidArrayCount( Array Array );
native(3107) static final function int DynVectorArrayCount( Array Array );
native(3108) static final function int DynPlaneArrayCount( Array Array );
native(3109) static final function int DynRotatorArrayCount( Array Array );
native(3110) static final function int DynCoordsArrayCount( Array Array );
native(3111) static final function int DynScaleArrayCount( Array Array );
native(3112) static final function int DynColorArrayCount( Array Array );
native(3113) static final function int DynBoundingBoxArrayCount( Array Array );
native(3114) static final function int DynBoundingVolumeArrayCount( Array Array );
Note that I have not yet checked if whether the baseclass for a struct is enough, say beeing able to use Array with DynVectorArrayCount. I also haven't checked whether the DynByteArrayCount also works with enums.

And now for some good news, we can use the operator system to wrap these all under
a common name:

Code: Select all

static final preoperator int DynArrayCount( Array            Array ) { return DynIntArrayCount           (Array); }
static final preoperator int DynArrayCount( Array          Array ) { return DynFloatArrayCount         (Array); }
static final preoperator int DynArrayCount( Array         Array ) { return DynStringArrayCount        (Array); }
static final preoperator int DynArrayCount( Array           Array ) { return DynNameArrayCount          (Array); }
static final preoperator int DynArrayCount( Array         Array ) { return DynObjectArrayCount        (Array); }
static final preoperator int DynArrayCount( Array           Array ) { return DynGuidArrayCount          (Array); }
static final preoperator int DynArrayCount( Array         Array ) { return DynVectorArrayCount        (Array); }
static final preoperator int DynArrayCount( Array          Array ) { return DynPlaneArrayCount         (Array); }
static final preoperator int DynArrayCount( Array        Array ) { return DynRotatorArrayCount       (Array); }
static final preoperator int DynArrayCount( Array         Array ) { return DynCoordsArrayCount        (Array); }
static final preoperator int DynArrayCount( Array          Array ) { return DynScaleArrayCount         (Array); }
static final preoperator int DynArrayCount( Array          Array ) { return DynColorArrayCount         (Array); }
static final preoperator int DynArrayCount( Array    Array ) { return DynBoundingBoxArrayCount   (Array); }
static final preoperator int DynArrayCount( Array Array ) { return DynBoundingVolumeArrayCount(Array); }
As this is just pure unrealscript, we can also just implement the operators into any other, even pure unrealscript class:

Code: Select all

static final preoperator int DynArrayCount( Array            Array ) { return Class'HXObject'.static.DynIntArrayCount           (Array); }
static final preoperator int DynArrayCount( Array          Array ) { return Class'HXObject'.static.DynFloatArrayCount         (Array); }
static final preoperator int DynArrayCount( Array         Array ) { return Class'HXObject'.static.DynStringArrayCount        (Array); }
static final preoperator int DynArrayCount( Array           Array ) { return Class'HXObject'.static.DynNameArrayCount          (Array); }
static final preoperator int DynArrayCount( Array         Array ) { return Class'HXObject'.static.DynObjectArrayCount        (Array); }
static final preoperator int DynArrayCount( Array           Array ) { return Class'HXObject'.static.DynGuidArrayCount          (Array); }
static final preoperator int DynArrayCount( Array         Array ) { return Class'HXObject'.static.DynVectorArrayCount        (Array); }
static final preoperator int DynArrayCount( Array          Array ) { return Class'HXObject'.static.DynPlaneArrayCount         (Array); }
static final preoperator int DynArrayCount( Array        Array ) { return Class'HXObject'.static.DynRotatorArrayCount       (Array); }
static final preoperator int DynArrayCount( Array         Array ) { return Class'HXObject'.static.DynCoordsArrayCount        (Array); }
static final preoperator int DynArrayCount( Array          Array ) { return Class'HXObject'.static.DynScaleArrayCount         (Array); }
static final preoperator int DynArrayCount( Array          Array ) { return Class'HXObject'.static.DynColorArrayCount         (Array); }
static final preoperator int DynArrayCount( Array    Array ) { return Class'HXObject'.static.DynBoundingBoxArrayCount   (Array); }
static final preoperator int DynArrayCount( Array Array ) { return Class'HXObject'.static.DynBoundingVolumeArrayCount(Array); }
So one could start from there and start to add structs declared in Actor, etc. but the approach is the same. It kinda sucks, but it should actually be good enough for a lot of tasks. I started to use it to specifcy an arbitrary amount of sounds in defaultproperties, to pick a random selected one out of it.

Also, one can always continue and start implementing += operators to add items, etc.

When you use dynamic arrays in native classes, you run into the issues with them not getting exported as TArrayNoInit, so you would end up having to maintain your constructors to deal with them, but you can hook UArrayProperty::ExportCppItem, while solves this issue in an automated way.

Code: Select all

//
// Did always export as TArray, but it needs to be either TArrayNoInit
// inside classes or the E_NoInit constructor needs to be called,
// as otherwise it would empty the array.
//
// Needless to say that manually calling E_NoInit isn't an option.
//
void UArrayPropertySubstitute::ExportCppItem( FOutputDevice& Ar ) const
{
      guard(UArrayPropertySubstitute::ExportCppItem);
      Ar.Log( (PropertyFlags&CPF_Parm) ? TEXT("TArray") );
      unguardobj;
}
Last edited by han on Tue Jul 02, 2019 6:47 pm, edited 1 time in total.
HX on Mod DB. Revision on Steam. Löffels on Patreon.
User avatar
han
Global Moderator
Posts: 686
Joined: Wed Dec 10, 2014 12:38 am

Re: Unfinished Language features.

Post by han »

While it is still to me pretty unclear how I would design the operators to work with the dynamic arrays, only one operator is dead clear to me, to be implemented in that way:

To use !MyArray as a shortcut for DynArrayCount(MyArray)==0

Code: Select all

// Shortcut for DynArrayCount(Array)==0.
static final preoperator bool !( out Array            Array ) { return DynIntArrayCount           (Array)==0; }
static final preoperator bool !( out Array           Array ) { return DynByteArrayCount          (Array)==0; }
static final preoperator bool !( out Array          Array ) { return DynFloatArrayCount         (Array)==0; }
static final preoperator bool !( out Array         Array ) { return DynStringArrayCount        (Array)==0; }
static final preoperator bool !( out Array           Array ) { return DynNameArrayCount          (Array)==0; }
static final preoperator bool !( out Array         Array ) { return DynObjectArrayCount        (Array)==0; }
static final preoperator bool !( out Array           Array ) { return DynGuidArrayCount          (Array)==0; }
static final preoperator bool !( out Array         Array ) { return DynVectorArrayCount        (Array)==0; }
static final preoperator bool !( out Array          Array ) { return DynPlaneArrayCount         (Array)==0; }
static final preoperator bool !( out Array        Array ) { return DynRotatorArrayCount       (Array)==0; }
static final preoperator bool !( out Array         Array ) { return DynCoordsArrayCount        (Array)==0; }
static final preoperator bool !( out Array          Array ) { return DynScaleArrayCount         (Array)==0; }
static final preoperator bool !( out Array          Array ) { return DynColorArrayCount         (Array)==0; }
static final preoperator bool !( out Array    Array ) { return DynBoundingBoxArrayCount   (Array)==0; }
static final preoperator bool !( out Array Array ) { return DynBoundingVolumeArrayCount(Array)==0; }
HX on Mod DB. Revision on Steam. Löffels on Patreon.
User avatar
han
Global Moderator
Posts: 686
Joined: Wed Dec 10, 2014 12:38 am

Re: Unfinished Language features.

Post by han »

I started using Num() instead of DynArrayCount(), which is shorter, and is somewhat in the line of what was done in the same way for string functions.
HX on Mod DB. Revision on Steam. Löffels on Patreon.
Post Reply

Return to “Unreal Engine 1 and 2”