Multiple SimConnect Clients - Weird Startup Behaviour

[DocMoriarty]

So meanwhile i have an exe (managed) and a dll (C++ native). Both are tested and usually work well. Usually i started my exe manually before i start FSX since it can run without FSX too. Unless connected to FSX the exe polls for a connection every soandsomany seconds.

When working on my dll i added an entry to dll.xml (prompt before load). After the dll was working fine, i thought i'd give it shot and started both, my exe first and then FSX with the dll. BOOM. FSX freezes in the main menu. Even when i declined to load the dll when asked it still kept freezing. Very weird. Only when removing the dll entry in dll.xml completely i was able to launch my exe before starting FSX.

After a lot of experimenting if found the following: I need to start FSX first if i want a dll client and the exe client afterwards, otherwise most likely something will crash. Using or not using SimConnect logging or diagnostic window didn't make a difference. I guess one or another made similar observations.

These instabilities should be fixed ASAP by ACES, since in a production scenario you can never say what a user will load first (unless you hide your exe and let it start by exe.xml but not explicitely from start menu).

 

[jcboliveira]

Never noticed that. I didn't experimented like you did but right now I have two simconnect dll: FSUIPC and Mytraffic and some own exe and never got a crash. Of course I can't control what the dll are doing since they are 3rd party but my app pools the connection like your does.

José

 

[Pete Dowson]

Never noticed that. I didn't experimented like you did but right now I have two simconnect dll: FSUIPC and Mytraffic and some own exe and never got a crash.

The problem of more than one SimConnect client initialising at roughly the same time and crashing FSX has been reported a lot on my Support forum and I've entered a bug for it with MS. More reports direct to tell_fs@microsoft.com would help wieght it in favour of an urgent fix though.

In your case Jose, the DLL your MyTrafficX loads is not truly a SimConnect client, I believe, as it only uses the DLL.XML to get itself loaded, just like the other FSX SDK DLLs.

To generate the problem reliably with an EXE is not easy as it is timing dependent. If you really want to crash FSX this way you would at least need to load the EXE via the EXE.XML file, and probably shuffle the position of FSUIPC4 in the DLL.XML.

The most common combination crashing FSX reliably is having FSUIPC4 load first then FSCopilot.DLL. If FSUIPC4 loads first it seems okay. I've also heard of the same problem with the TrackIR exe and FSCopilot, and so on .... pretty much any combination of two or more, and the more the worse.

Even if FSX struggles past this initialisation problem, busy multiple SimConnect clients remain precarious in FSX -- crashes, stoppages, and so on can and often do follow during the FSX session at any time later. The most common is that one or other of the clients stops responding (stops getting SimConnect data) after an "I/O Error" is logged by SimConnect. Seems that a busy TCP/IP exchange combined with a busy FSX generates some sort of error in the TCP/IP stack -- presumably a timeout or queuing problem. Can't imagine it can be a real I/O error when everything is not only in the one PC, but in the one Process.

I am campaigning for the internal SimConnect to FSX communication to be re-coded to use straight messages, bypassing all this TCP/IP nonsense. I can understand the use of TCP/IP for external clients, but for internal ones it simply makes no sense to me and is the cause of a lot of the problems being reported, as well as about a 10-15% overhead in running FSUIPC4, something I've *never* had in the previous three versions of FS because I've kept the code so efficient. :-(

Regards

Pete

 

[DocMoriarty]

That just confirms my observations. It didn't always crash when my exe was up first. But my exe polls every 2 seconds for a connect until it's established and that gives a high chance for the crash symptoms to occur.

One weird thing though was, it kept crashing when i said at the FSX load dll prompt not to load it. Only disabling or removing the entry from dll.xml "healed" the symptoms (or loading the exe when FSX finally shows it's main menu). So there is also some bug directly in the load dll logic in FSX.

I don't think that TCP/IP is the real problem, there is some kind of race condition in FSX initialization that is not handled properly i guess. I mean by that amount of threads opened by FSX it's not really tremendously surprising.

I think though that with using TCP/IP inside one machine the major backdraw are absolute completely unnecessary problems with firewalls, anti virus stuff etc, not to mention windows' own security nightmare.

But meanwhile i started to write my own managed wrapper since i ran in real big trouble with managed SimConnect regarding client data areas and a few other things that i dislike in managed SimConnect (like the bright idea of using System.Enum types for ID's on input and events return Integers, now try to compare those two, a few missing functions necessary for debugging like GetLastPackageID(), no handle to native SimConnect for emergency cases, no enum types defined for some stuff (dwID for example), splitting events into several event functions depending on return type to save a datatype cast thus forcing the programmer to tear apart central program logic and to guess which of the events might actually return what you need, and so on and so forth ...). It's working o far ^^

 

[Pete Dowson]

One weird thing though was, it kept crashing when i said at the FSX load dll prompt not to load it. Only disabling or removing the entry from dll.xml "healed" the symptoms (or loading the exe when FSX finally shows it's main menu). So there is also some bug directly in the load dll logic in FSX.

Yes, of course. In fact the very first reproducible crash of this type was when FSUIPC4 or another DLL, which did NOT generate a Security or Load dialogue confirmation request, was loading when simultaneously another DLL or EXE was generating a Security warning or Load dialogue question. The details of that particular type of crash were submitted to MS back in October As far as I can recall.

I don't think that TCP/IP is the real problem, there is some kind of race condition in FSX initialization that is not handled properly i guess.

Correct. THAT problem isn't, as far as I can tell, anything to do with TCP/IP, which really isn't particularly active in at least one of the Clients in any case. Somrthing in the DLL/EXE loading process is causing corruption somewhere wildlyy inappropriate in FSX's loaded data -- the crashes most often appear to be deep inside TERRAIN.DLL, or all the unlikely places! Not only that, but sometimes this corruption may not cause an immediate crash, but one 20 or 30 minutes later!

I think though that with using TCP/IP inside one machine the major backdraw are absolute completely unnecessary problems with firewalls, anti virus stuff etc, not to mention windows' own security nightmare.

Yes, but also performance, which concerns me even more. And even when the security settings are such that the communications are not actually blocked, they seem to often be delayed. With some security products we are noticing increasing delays in the transmission from Simconnect to FSX of events sent via TransmitClientEvent. Controls for flight, for instance, can suffer a latency starting at almost zero and building up to as much as 30 seconds. I've no idea how the TCP/IP setup operates such queues, but it seems it does and can do, and with TCP being used, which guarantees delivery and in the correct order (unlike UDP) it may not be so surprising if there are delays and overherads someplace, maybe in the FSX end of SimConnect simply not processing them fast enough in busy periods in an already overworked processor.

Regards

Pete

 

[DocMoriarty]

Yes i notice quite a bit latency at times when flying when my client is running. Not always, it differs. Joystick inputs often react quite a bit more when using my client. Weird is that the framerate is ok, don't notice much difference. I guess frames are produced in another thread than calculation of the simulation.
It depends though for example on whether i use a debug version or the release version. I use cpu stats in my log and when my client is up and running it takes around 0,03% cpu time (kernel+user), so that can't really explain the latency. I somehow have the suspicion that it has to do with synchronization of threads, that is there is a point in fsx where the sim engine synchronizes with the simconnect server engine and waits until sending and receiving stuff is done for that "cycle". Never had such long 30s delays so far.
I have no firewalls running though and no anti virus software, html blockers or other stuff that smokes the TCP/IP stack. Research continues.

Merry Christmas to everyone!