logo
Main

Forums

Downloads

Unreal-Netiquette

Donate for Oldunreal:
Donate

borderline

Links to our wiki:
Wiki

Walkthrough

Links

Tutorials

Unreal Reference

Usermaps

borderline

Contact us:
Submit News
Page Index Toggle Pages: [1] 2 3  Send TopicPrint
Very Hot Topic (More than 25 Replies) Is there a proper fix for TArray<Byte> linkage? (Read 6861 times)
[]KAOS[]Casey
Developer Team
Betatester
Offline


nedm

Posts: 3196
Joined: Aug 7th, 2011
Gender: Male
Is there a proper fix for TArray<Byte> linkage?
Sep 6th, 2016 at 8:09am
Print Post  
I want/need to use "appLoadFileToArray" for loading a .dds into memory so I can check if it's a proper magic number, and load it into a buffer and so on..

Is there some proper way to trick the compiler to link properly for UT binaries?

If not, a workaround would suffice for this instance.
  
Back to top
 
IP Logged
 
han
Global Moderator
Unreal Rendering Guru
Developer Team
*****
Offline


Oldunreal member

Posts: 565
Location: Germany
Joined: Dec 10th, 2014
Gender: Male
Re: Is there a proper fix for TArray<Byte> linkage?
Reply #1 - Sep 6th, 2016 at 8:22pm
Print Post  
Whats the problem with it? Are you sure it's not caused by the msvc version you are using? At least for building ALAudio I ended in removing the ENGINE_API specifier on FSoundData to get around some linker issues.
  

HX on Mod DB. Revision on Steam.
Back to top
 
IP Logged
 
han
Global Moderator
Unreal Rendering Guru
Developer Team
*****
Offline


Oldunreal member

Posts: 565
Location: Germany
Joined: Dec 10th, 2014
Gender: Male
Re: Is there a proper fix for TArray<Byte> linkage?
Reply #2 - Sep 6th, 2016 at 8:23pm
Print Post  
Despite.. why don't you use a UFactory?
  

HX on Mod DB. Revision on Steam.
Back to top
 
IP Logged
 
[]KAOS[]Casey
Developer Team
Betatester
Offline


nedm

Posts: 3196
Joined: Aug 7th, 2011
Gender: Male
Re: Is there a proper fix for TArray<Byte> linkage?
Reply #3 - Sep 6th, 2016 at 9:06pm
Print Post  
Because I'm implementing DDS importing by effectively creating my own UFactory, UT99 doesn't have DDS importer to my knowledge.

I'm also absolutely sure it is caused by MSVC, which is why I'm asking here.

It's the same old song and dance -- constructors and destructors, as well as other miscellaneous functions aren't linking.

I'll probably just end up coding my own way to load the file into memory instead of using unreals implementation.
  
Back to top
 
IP Logged
 
han
Global Moderator
Unreal Rendering Guru
Developer Team
*****
Offline


Oldunreal member

Posts: 565
Location: Germany
Joined: Dec 10th, 2014
Gender: Male
Re: Is there a proper fix for TArray<Byte> linkage?
Reply #4 - Sep 6th, 2016 at 10:16pm
Print Post  
Can you post the link erorrs? I'm not at home and right now have no access to msvc 2013...  I suggested the UFactory as you already get the file data passed in and you wont need to use the appLoadfimetoarray this way. Otherwise you could use the filemanager to geg the filesize, appalloc and Ar.Serialize( ptf, size ). Or directly create the appropreciate structures for serializing the header.
  

HX on Mod DB. Revision on Steam.
Back to top
 
IP Logged
 
[]KAOS[]Casey
Developer Team
Betatester
Offline


nedm

Posts: 3196
Joined: Aug 7th, 2011
Gender: Male
Re: Is there a proper fix for TArray<Byte> linkage?
Reply #5 - Sep 6th, 2016 at 10:36pm
Print Post  
Oh interesting,

I'll try that with the UFactory.

I will post the linker errors as soon as I'm home.
  
Back to top
 
IP Logged
 
[]KAOS[]Casey
Developer Team
Betatester
Offline


nedm

Posts: 3196
Joined: Aug 7th, 2011
Gender: Male
Re: Is there a proper fix for TArray<Byte> linkage?
Reply #6 - Sep 7th, 2016 at 12:21am
Print Post  
Code
Select All
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: void __thiscall TArray<unsigned char>::Remove(int,int)" (__imp_?Remove@?$TArray@E@@QAEXHH@Z)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: void __thiscall TArray<unsigned short>::Remove(int,int)" (__imp_?Remove@?$TArray@G@@QAEXHH@Z)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: int __thiscall TArray<unsigned char>::AddItem(unsigned char const &)" (__imp_?AddItem@?$TArray@E@@QAEHABE@Z)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: int __thiscall TArray<unsigned char>::Add(int)" (__imp_?Add@?$TArray@E@@QAEHH@Z)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: unsigned char & __thiscall TArray<unsigned char>::operator()(int)" (__imp_??R?$TArray@E@@QAEAAEH@Z)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: __thiscall TArray<unsigned char>::~TArray<unsigned char>(void)" (__imp_??1?$TArray@E@@QAE@XZ)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: __thiscall TArray<unsigned char>::TArray<unsigned char>(void)" (__imp_??0?$TArray@E@@QAE@XZ)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: unsigned short & __thiscall TArray<unsigned short>::operator()(int)" (__imp_??R?$TArray@G@@QAEAAGH@Z)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: __thiscall TArray<unsigned short>::~TArray<unsigned short>(void)" (__imp_??1?$TArray@G@@QAE@XZ)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: __thiscall TArray<unsigned short>::TArray<unsigned short>(void)" (__imp_??0?$TArray@G@@QAE@XZ) 

  
Back to top
 
IP Logged
 
[]KAOS[]Casey
Developer Team
Betatester
Offline


nedm

Posts: 3196
Joined: Aug 7th, 2011
Gender: Male
Re: Is there a proper fix for TArray<Byte> linkage?
Reply #7 - Sep 7th, 2016 at 1:14am
Print Post  
Looks like I might just have to install VC6 on a VM for this. Mips uses a few tarrays.
  
Back to top
 
IP Logged
 
Masterkent
Developer Team
Offline



Posts: 1330
Location: Russia
Joined: Apr 5th, 2013
Gender: Male
Re: Is there a proper fix for TArray<Byte> linkage?
Reply #8 - Sep 7th, 2016 at 9:22am
Print Post  
[]KAOS[]Casey wrote on Sep 7th, 2016 at 12:21am:
Code
Select All
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: void __thiscall TArray<unsigned char>::Remove(int,int)" (__imp_?Remove@?$TArray@E@@QAEXHH@Z)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: void __thiscall TArray<unsigned short>::Remove(int,int)" (__imp_?Remove@?$TArray@G@@QAEXHH@Z)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: int __thiscall TArray<unsigned char>::AddItem(unsigned char const &)" (__imp_?AddItem@?$TArray@E@@QAEHABE@Z)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: int __thiscall TArray<unsigned char>::Add(int)" (__imp_?Add@?$TArray@E@@QAEHH@Z)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: unsigned char & __thiscall TArray<unsigned char>::operator()(int)" (__imp_??R?$TArray@E@@QAEAAEH@Z)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: __thiscall TArray<unsigned char>::~TArray<unsigned char>(void)" (__imp_??1?$TArray@E@@QAE@XZ)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: __thiscall TArray<unsigned char>::TArray<unsigned char>(void)" (__imp_??0?$TArray@E@@QAE@XZ)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: unsigned short & __thiscall TArray<unsigned short>::operator()(int)" (__imp_??R?$TArray@G@@QAEAAGH@Z)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: __thiscall TArray<unsigned short>::~TArray<unsigned short>(void)" (__imp_??1?$TArray@G@@QAE@XZ)
1>TextureMergerMain.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: __thiscall TArray<unsigned short>::TArray<unsigned short>(void)" (__imp_??0?$TArray@G@@QAE@XZ) 


The first question is: why are definitions of all these template specializations have to be obtained from dll? Why do you suppress implicit instantiation of these specializations?

The second question is: what code do you use to instantiate and export these specializations (for the given lists of template arguments) in that dll?
  
Back to top
 
IP Logged
 
han
Global Moderator
Unreal Rendering Guru
Developer Team
*****
Offline


Oldunreal member

Posts: 565
Location: Germany
Joined: Dec 10th, 2014
Gender: Male
Re: Is there a proper fix for TArray<Byte> linkage?
Reply #9 - Sep 7th, 2016 at 10:59am
Print Post  
[]KAOS[]Casey wrote on Sep 7th, 2016 at 1:14am:
Looks like I might just have to install VC6 on a VM for this. Mips uses a few tarrays.

Fyi, msvc6 after install runs without issues even on Win10.

I could sent you my headers I use for ut, which has has quite some changes done in UnTemplate.h. might work for you.

Though I saw some post/thread by Hidor about another way fixing it some time ago, but afaik it wasnt on oldunreal, but this ut site.. I sent Smirftsch the link :p

You could also try removing the ENGINE_API on FMipMapBase or FMipMip... as this worked for FSojndData.
  

HX on Mod DB. Revision on Steam.
Back to top
 
IP Logged
 
Masterkent
Developer Team
Offline



Posts: 1330
Location: Russia
Joined: Apr 5th, 2013
Gender: Male
Re: Is there a proper fix for TArray<Byte> linkage?
Reply #10 - Sep 7th, 2016 at 12:34pm
Print Post  
ENGINE_API is just a macro which is expanded to __declspec(dllimport). When applied to a class, it is automatically applied to all its member functions. When applied to a member function of a class template, implicit instantiation of definition of the given member function will be suppressed, because the definition of a member function of a class template specialization is then supposed to be obtained from an external library. If external library does not provide the definition having exactly the same signature (which is distinguished by mangled function name), linking to the function fails.

When a member function of a class template is declared with dllexport (which can be achieved by declaring the whole class with dllexport), presence of the generic definition of the function is not sufficient for exporting any of its specializations. If you want to export a specialization (an instance) of a member function of a class template with a particular template-argument-list, you have to instantiate it. I don't remember if implicit instantiation is enough, but explicit instantiation should work. For a particular function it would be something like this:

Code (C++)
Select All
template void TArray<unsigned char>::Remove(int, int);
// instantiates the definition of the member function of TArray<unsigned char> from the generic definition 


It is also possible to explicitly instantiate the whole class:

Code (C++)
Select All
template class TArray<unsigned char>;
// instantiates the whole class TArray<unsigned char> from the class template definition 


Not sure if __declspec(dllexport) has to be present in either case.

Now about removing ENGINE_API. By removing __declspec(dllimport) for a particular function you allow implicit instantiation of the function definition from the generic definition available in the corresponding header file (so the current module will use its own definition of the function rather than expect that an external library will provide it). Then you must ensure that if several modules operate on the same object, the operations are consistent across modules. In particular, all modules shall use the same heap for memory allocations and deallocations.
  
Back to top
 
IP Logged
 
han
Global Moderator
Unreal Rendering Guru
Developer Team
*****
Offline


Oldunreal member

Posts: 565
Location: Germany
Joined: Dec 10th, 2014
Gender: Male
Re: Is there a proper fix for TArray<Byte> linkage?
Reply #11 - Sep 7th, 2016 at 2:27pm
Print Post  
In any case the templates in UnTemplate.h use GMalloc for allocation/deallocation so this is not an issue at all. Also there are a lot of non UObject based classes inside the headers where all code you use or usually use is present, so removing the CORE_API macro on FString allows you too fix the plenty of bugs related to empty strings, have a more common plattform across different ue1 games.

In any case I prefer using MSVC6 for UE1 stuff...
  

HX on Mod DB. Revision on Steam.
Back to top
 
IP Logged
 
Masterkent
Developer Team
Offline



Posts: 1330
Location: Russia
Joined: Apr 5th, 2013
Gender: Male
Re: Is there a proper fix for TArray<Byte> linkage?
Reply #12 - Sep 7th, 2016 at 2:52pm
Print Post  
han wrote on Sep 7th, 2016 at 2:27pm:
In any case I prefer using MSVC6 for UE1 stuff...

I never worked on native mods for Unreal/UT, so I can't tell how much the existing code base of U/UT relies on the specific behavior of MSVC 6. Nevertheless, I worked with MSVC 6 per se, and, from what I remember, it's an old piece of sh... with a lot of bugs and poor support of C++ features (even for '98 year) - especially with regard to templates. Nowadays I can't consider it other than as a museum specimen. Modern compilers offer much better language support and quality of generated code.
  
Back to top
 
IP Logged
 
han
Global Moderator
Unreal Rendering Guru
Developer Team
*****
Offline


Oldunreal member

Posts: 565
Location: Germany
Joined: Dec 10th, 2014
Gender: Male
Re: Is there a proper fix for TArray<Byte> linkage?
Reply #13 - Sep 7th, 2016 at 4:08pm
Print Post  
As UE1 sort of contains it own standard library the sort of problematic template implementation is no issue. As for compiler bugs, I never ran into a single one so far, for uc i did.

The IDE itsself is imho the best vc ide ms ever created. And it's incredible fast whereas msvc 2013 is slow as hell.

As for generated  code, you don't gain anything from compiling old code with it. A lot of the code uses a lot of branching as old optimizagion, while for really gaining perf one need to migrage the code to less branching and especially staying more local in mem and also especially remove a lot of asm opts and CORE_API stuff in UnMath.h. But still for make efficient use of SSE you do want a broader alignment than what your data usually gets passed in for you. What really gets a lot faster is using msvc2013 acos function, but this in turn will use a different runtime and you might get troubles with the internal states of the runtimes, etc. Afaik there was a nice article about the runtime problematic linked on the libpng page. 

So in general you just gain problems when using a newer msvc version for ue1 games build with vc6 and virtually gain nothing.
  

HX on Mod DB. Revision on Steam.
Back to top
 
IP Logged
 
Masterkent
Developer Team
Offline



Posts: 1330
Location: Russia
Joined: Apr 5th, 2013
Gender: Male
Re: Is there a proper fix for TArray<Byte> linkage?
Reply #14 - Sep 7th, 2016 at 9:03pm
Print Post  
han wrote on Sep 7th, 2016 at 4:08pm:
As UE1 sort of contains it own standard library the sort of problematic template implementation is no issue.

That library covers very few needs, and, for example, where possible, I'd rather use C++11/14 std::vector instead of nonsense like TArray.

Quote:
As for compiler bugs, I never ran into a single one so far

You probably used a very small subset of language features. I faced with compiler bugs regularly. Sometimes compiler just crashed when trying to compile a valid code. In some cases I got hard-to-find bugs in compiled programs due to erroneously performed overload resolution or missing initialization of objects that had to be done according to the language rules.

Quote:
As for generated  code, you don't gain anything from compiling old code with it.

I don't think so. Optimizers were improved a lot since those ancient times. E.g. new compilers can inline more functions. IIRC, link time code generation was not supported by VC++ 6 at all. Better inlining alone already can give a noticeable performance boost.

Quote:
So in general you just gain problems when using a newer msvc version for ue1 games build with vc6 and virtually gain nothing.

I wonder what problems could arise there. Does that old code base rely on specific compiler bugs and/or undocumented behavior so much or what?
  
Back to top
 
IP Logged
 
Page Index Toggle Pages: [1] 2 3 
Send TopicPrint
Bookmarks: del.icio.us Digg Facebook Google Google+ Linked in reddit StumbleUpon Twitter Yahoo