1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Feature request: MCX model 'component' exporter

Discussion in 'ModelConverterX' started by Bungo, 12 Nov 2017.

  1. Bungo

    Bungo

    Joined:
    26 Jul 2008
    Messages:
    107
    Country:
    australia
    It would be nice to be able to split up a scene in MCX so that you can individually export the elements (individual buildings) to individual MDL files all use the same materials. This is so that materials can be sure to share exactly the same properties where multiple buildings share the same texture sheet (in FSX), and not create a separate instance of each sheet in memory for the different models. ie as a way of ensuring that all the texture sheet properties are identical and not subtly different in some way by proccessing them separately.

    Generally speaking...

    [​IMG]

    1. Import a scene of many buildings that share a collection of textures, not necessarilly placed at any locaton geographically or in relation to each other.
    2. Optimize the textures and export them.
    3. Select each building and export to MDL format of choice. Or have a function that that selects each isolated or disconnected structure, and in-turn moves the reference point to a corner of it, and exports each to a file with a generic name that you can change yourself later.
    Basically, the reverse of the scenebuilder wizard. Deconstruct models so that all the model elements can be placed separately, which might be geographically dispersed and with different altitudes of terrain under them, to be sure to have identical material specs for each so optimising.

    Don't know how to keep guid's of the comoponents consistent when you export the scene many times (ie when testing and developing).

    cheers

    Braedon
     
  2. arno

    arno Administrator Staff Member FSDevConf team Resource contributor

    Joined:
    28 May 2004
    Messages:
    24,831
    Country:
    netherlands
    Hi,

    My main concern is how do you know which polygons form a structure? Especially if they share the same texture that information might be lost. I would have to check.
     
  3. Bungo

    Bungo

    Joined:
    26 Jul 2008
    Messages:
    107
    Country:
    australia
    There may be a pre-requisite that they not be connected within the model? I would guess that would cover the majority of cases. How you test for that?

    Thanks

    Braedon
     
  4. arno

    arno Administrator Staff Member FSDevConf team Resource contributor

    Joined:
    28 May 2004
    Messages:
    24,831
    Country:
    netherlands
    Maybe each building has a parent node in the COLLADA file. In that case I could split on that.
     
  5. =rk=

    =rk=

    Joined:
    28 Nov 2009
    Messages:
    1,609
    Country:
    us-washington
    Consider that you are not able to click, highlight or interact with any individual part by using the cursor within an MCX scene. All you are able to do, in fact, is to loosely "grab" the model and rotate it - and you know, Arno, for each building to have a parent node, the author would have to define one, otherwise there would be nothing to distinguish one extrusion from any other.
    The fact that Arno created so thoroughly convincing a render and allowed the user to rotate the view, probably confuses many people into thinking it is more powerful than it is, it did me. There is no spacial definition of the model in relation to the cursor, that relationship is part of 3d modelling software. MCX cannot click, manipulate and move individual parts. This is one of the reasons Autodesk 3ds Max costs several thousand US dollars and MCX is free. In the "eyes" of MCX, there is no "collection" of buildings. To MCX, every model is a single tapestry, each element of that tapestry defined only by it's own geographical coordinates, it's relation to the center and material settings. There is no "edge," no "polygons end here," kind of thing, our eyes do that and even then it is a sort of optical illusion, the idea that one cloud of mathematically assigned polygons is "different" from another is almost philosophical and you would probably need Watson installed to get a computer to make the distinction any more satisfyingly than we do.

    There is 3d software, between itself and MCX, that can accomplish more or less what you describe. The technique places the onus of the work on the artist and not Arno. Coincidentally, this will work in the software pictured above, it may also work with others.
    So you have a city scene, you have five distinct buildings and they each call to the same texture sheet, using a different mapped section. Save you model in Sketchup, Call it "5city." Now delete all except one building, If you feel you may need to edit this sub-component, save it as "5city1," otherwise export it with that name. Perform the same procedure with each building, sequentially naming them, 5city1 through 5city5. (I never start with "0" and this sometimes does cause confusion.)
    Now import 5city1 into MCX and perform your operations. 5city1, like all models, will retain its placement orientation in relation to the rest of the model, unless you center it using MCX.
    As the artist, it is and should remain your responsibility to ensure that texture sheet properties are exactly as you intend, it is not that hard to do and they do not change arbitrarily. Arno has already gone to the trouble to provide a material attributes template to which you can assign the exact profile you desire and you can create as many of there profiles as you like.

    After editing and compiling each of the sub-components, you now have a collection of 5 distinct models that all call the same texture file. If you want to re-combine them with MCX, you can use the join tool.

    If Arno could find a way to identify the Sketchup "group," or "component" labels, those would generally conform to distinct sub models. Group in particular, in very many cases will include one exact and entire sub model, especially if artists were prepared for it. However this would require a .skp importer because I doubt this group information is preserved into any other format.
     
  6. Bungo

    Bungo

    Joined:
    26 Jul 2008
    Messages:
    107
    Country:
    australia
    If when preparing the sketchup file for export (if using sketchup), if you convert each individual structure into a component, you create a <node></node>entry that seems to enclose the entire object, and you give it a name that can be used when later exporting the components. It may not be as obvious if you just create a scene with a bunch of buildings.

    @=rk=, it's not just the texture image itself that I am looking at ensuring is the same but all of the other parameters that have to exactly match for FSX to consider it to be the same and not define a separate material and a duplicate of the image. Get one that is not quite the same and you don't have the desired result. Manually checking for this would be extremely laborious, and if you can only load one model at a time, extremely impractical. Thus, the simplest way to get this right would be to take a scene and break it up, or have MCX open multiple models, let you optimise them together and save them all again. The latter could be an option too, depending on which is more difficult to program. That method solves the issue detecting the different models/components in the DAE file.

    cheers

    Braedon
     
    Last edited: 12 Nov 2017
  7. Pyscen

    Pyscen Resource contributor

    Joined:
    13 Aug 2006
    Messages:
    1,087
    Country:
    us-texas
    Hello...

    I understand what Braedon is saying, but at the same time, I know that this type of modeling (for lack of a better term, "clustering"), clustering many models in this way, in SketchUp or otherwise, could lead to many problems if it isn't separated to begin with (such as: floating models, memory drain and when it should appear to the user etc.).

    If the reason for designing in this fashion is to ensure that the multiple structures or models are using identical materials or textures then what's needed is a setting (within MCX) to accommodate it. But I don't think having MCX "uncluster" them would be the best possible way in doing that to begin with. This maybe is more of a "Library Creator XML" feature. Maybe within a Library, one could check to see that all textures within a given group of models are the same (as in settings and materials used).

    Otherwise, what @=rk= is saying is correct, the separating the models out etc within SketchUp, as well as in saving texture settings within MCX.
     
    Last edited: 12 Nov 2017
  8. =rk=

    =rk=

    Joined:
    28 Nov 2009
    Messages:
    1,609
    Country:
    us-washington
    The term <node></node>entry does not apply to Collada, .kmz, or .skp files. Despite what else it seems it definitely seems confusing, it appears to apply to javascript or something. So you might also be asking Arno to create a .skp importer, which I am sure he's already looked into. Reiterating my original reply, if you convert each individual structure into a [sub-model exported] component, then there is no requirement for a specific importer that identifies sketchup groups and components.
    This implies uncertainty about material attributes, as they affect only mapped textures, or colored polygons. A firm grasp of this will insure there is never any need or concern over duplicate textures.
    It is extremely impractical to apply Arno's finite time and infinite talents to creating shortcuts for a very practical learning curve at so very fundamental a level.
    The optimization function itself is a shortcut and when you learn of the inherent limitations, if your skill progresses to the point where such details matter, you will cease using the optimizer and recognize it as a tool to help the learning process. If you build your models properly, you know exactly what the material attributes are before they get to MCX and any that have to be edited.
    You have been offered techniques to use existing tool to accomplish EXACTLY what you seek and trust me, this pattern is mirrored in commercial scenery releases. Copying the pro's, I've created entire cities using Sketchup, MCX and fewer than 10 texture files. If anything like what you suggest could improve my workflow, I'd be all over it.

    [​IMG]

    Please, put nose to grindstone and come up with really interesting features for Arno to give us, thanks.
     
    Henk Schuitemaker likes this.
  9. arno

    arno Administrator Staff Member FSDevConf team Resource contributor

    Joined:
    28 May 2004
    Messages:
    24,831
    Country:
    netherlands
    To keep things simple, would you have a sample file with some clustered objects? Then I can check if there is an easy way to split them on import. If not, it would have to be done in SketchUp already.
     
  10. Bungo

    Bungo

    Joined:
    26 Jul 2008
    Messages:
    107
    Country:
    australia
    @arno see attachment for the requested examples.

    As far as I can tell (at least when exporting from Sketchup), if you create a component in sketchup from what you want to be one object, it creates 2 sections:

    Code:
        <library_visual_scenes>
            <visual_scene id="ID2">
                <node name="SketchUp">
                    <node name="skp_camera_Last_Saved_SketchUp_View">
                        <matrix>-0.06891823 -0.4333421 0.8985905 36125.5 0.9976223 -0.02993635 0.06207687 3728.313 -3.122502e-17 0.9007322 0.434375 15782.53 0 0 0 1</matrix>
                        <instance_camera url="#ID1" />
                    </node>
                    <node id="ID3" name="instance_0">
                        <matrix>-0.8766083 -0.4812047 0 10084.13 0.4812047 -0.8766083 0 1417.601 0 0 1 0.3466252 0 0 0 1</matrix>
                        <instance_node url="#ID4" />
                    </node>
                    <node id="ID1995" name="instance_1">
                        <matrix>-0.7087575 -0.7054522 0 -856.1168 0.7054522 -0.7087575 0 5041.066 0 0 1 0 0 0 0 1</matrix>
                        <instance_node url="#ID1996" />
                    </node>
                    <node id="ID2290" name="instance_2">
                        <matrix>-0.6204628 0.7842359 0 -986.3656 -0.7842359 -0.6204628 0 9360.148 0 0 1 0.3466252 0 0 0 1</matrix>
                        <instance_node url="#ID2291" />
                    </node>
                </node>
            </visual_scene>
        </library_visual_scenes>
    
    
    and

    Code:
       <library_nodes>
            <node id="ID4" name="IXL-WHARF">
                    .....
                    geometry definitions
                    .....
            </node>
            <node id="ID1996" name="HGC">
                    .....
                    geometry definitions
                    .....
            </node>
            <node id="ID2291" name="Woolstore_Appts">
                    .....
                    geometry definitions
                    .....
            </node>
    etc...
    And another 2 nodes, one to enclose the entire scene, and one to save the camera view in sketchup.

    When the same scene is saved with all of the components exploded into their constituent parts, you only get the 2 other nodes for scene and camera definitions, and a great many geometry definitions (at a quick glance).

    Attached are 2 DAE files zipped up of the same scene for the above example.

    I don't know if other package exporters from Blender, GMAX, and 3DS Max to DAE handle this the same, but this is how Sketchup seems to do it.

    Exactly how we get to the end result of having multiple models sharing textures/materials consistently and easilly without trolling through hundreds of parameters to ensure consistency (which sketchup is...well...sketchy at and doesn't expose to the designer in any detail), doesn't really matter:
    • Load multiple models into memory simultaneously, collectively optimise the textures, and re-export them all, no problem.
    • Load one larger scene of model structures into memory, optimise the textures, and explode the scene into diffrerent MDL files for individual placement, no problem.
    • Using Library Creator (which doesn't have a very complex GUI so would need all that created), no problem.
    • Some other clever, simple, and time saving method that I've not thought of, why not!

    Just for the record, because the pushback I'm feeling from some (not just in this thread BTW) is beginning to sound like I asked Arno to bury my sister in his back yard rather than possibly consider a feature request:

    Arno saw fit to give his time (presumably from a request from the community) to have a function to aggregate many models and optimise materials/textures and export them as one model. Now, that is OK if all the models are on the flat, close to each other, or everyone uses exactly the same terrain spacing and files. All I requested was a method of doing the reverse, for many of the same reasons, and to cover the situations that don't match the criteria above.

    Also, Sketchup doesn't give a lot of detail about it's materials or have a dedicated FSX/P3D plugin to play with every setting for FSX like 3DS Max has. There are a great many ways that it changes subtle details that only come out when exported and compared. Thus, MCX is the perfect tool to correct for this, and many of the features MCX has are specifically designed to overcome the shortcomings of the free tools people use that FSX never intended us to use for development. I don't see how this is any different.

    If Arno has higher priorities (which I'm sure there are), or can see how this can be achieved easilly another way, then equally, not a problem.

    But on that last count, I'll let Arno make that call, I don't see why others feel the need to do that for him. Given that Arno is asking me for more information, I don't see why others are taking offence at me requesting features just because they don't personally have the same issues. All of our workflows, and skills, and things we think are important in our projects are different.

    I have the utmost respect for Arno and what he does for this community, and the time and dedication he puts in to creating tools that have filled long standing and gaping holes in the capabilities of the official tools. Thus, I'm also old enough and ugly enough to handle it if Arno declines my request too. I would suggest that anyone else making requests should (if they don't already) have the same attitude. I'm sure that the vast majority of those making feature requests already do.

    Regards

    Braedon
     

    Attached Files:

    Last edited: 13 Nov 2017
  11. =rk=

    =rk=

    Joined:
    28 Nov 2009
    Messages:
    1,609
    Country:
    us-washington
    These are composition instructions for models saved as Sketchup "scenes," whereby the workspace view cycles from different camera angles. If one could imagine a complex sample of 3d geometry, from which one might want to display highly detailed or accurately reproduced features, one might want to compose a model with a view that cycles through several composed angles. The scene feature is unique to Sketchup and also has nothing to do with individual components beyond arranging the POV in relation to them.


    There is a very finite number of material attributes for you to troll and it is far lower than 100.
    It is clear you express benign intentions with no real grasp of what's involved. It is as if you've seen a thunderstick for the first time and understandably want one of your own. You give Arno a snippet of scene composing xml, a collada model that he will have to take the time to open, play with and compose you a very polite forum post about how you are so great even though it won't work - and expect him to make an entirely new functionality in MCX for it. Furthermore, the tools to accomplish what you intend already exist. You would want to make a really good case for re-inventing the wheel, perhaps you have.

    Sketchup was not written to accommodate FSX, nor was 3ds Max. Neither software has any native ability to make simulator specific changes, this is accomplished through plug-in integration with the SDK. You could write a similar plug-in for Sketchup, I know I discovered one from the Train Sim genre that automatically made LOD's and GUID's, if I am not mistaken; possibly it only identified reversed faces, which is more important in MSFS simulators than in other 3d rendered environments.
    The same might be said of an airplane cockpit by an uninitiated observer. Sketchup is incredibly powerful, versatile and intuitive. By versatile, it can perform very advanced 3d modelling, as well as rewardingly allow a new user to build skill. It does this by "allowing mistakes." 3ds Max, by contrast, is very rigid and will perform specific advanced operations quite well. One would want to educate one's self about these operations because trial and error would be unrewarding, at best. Sketchup is very subtle in it's indications, however it does not change things arbitrarily and your skills will benefit from becoming more familiar with the software. Eventually, you will likely follow the path of so many others, including myself, who have discarded polygon color in favor of mapped textures and upon that occasion the material optimizer pane will have very little relation to your work.

    EDIT: Looking back, I know my style of communication can cause people to feel defensive and that is not my intent, if I've done so I apologize. I really do want to encourage you and everyone to develop, to explore, to learn and share things. I hope you can help me help you to do this.
     
    Last edited: 13 Nov 2017
    Henk Schuitemaker likes this.
  12. GaryGB

    GaryGB

    Joined:
    23 Dec 2005
    Messages:
    3,685
    Country:
    us-illinois
    Hi Braedon:

    I'd like to better understand your ideas and goals for scenery building in the context of this discussion, which IIUC, may also be topically-related to a latter sub-topic that you had opened within another prior discussion here: :scratchch

    http://www.fsdeveloper.com/forum/th...-point-of-imported-models.441368/#post-784466


    ...and which may also have been discussed in the is thread from last Spring:

    http://www.fsdeveloper.com/forum/th...e-models-shared-consolidated-textures.439857/


    Your ZIP attachment above contains no texture Materials. :alert:

    Would you please export from Sketchup, a ZIP-ped KMZ file and attach / link to it, instead of the ZIP above ? :pushpin:


    FYI: A Google Earth KMZ file is a ZIP file which contains a Collada *.DAE file of the 3D model, as well as all pertinent Texture images, and Material files which define their Material Properties, their UVW mapping, and their Geo-location (Geo-referencing) information.


    Thanks for sharing your inquiry for consideration and discussion to explore possible ways of facilitating scenery development. :)


    PS: Please tell us which numeric version of Sketchup you are using.

    GaryGB
     
    Last edited: 14 Nov 2017
  13. Bungo

    Bungo

    Joined:
    26 Jul 2008
    Messages:
    107
    Country:
    australia
    I have a large urban area to cover with a combination of AGN houses, with mostly generic medium/industrial/larger buildings and custom models where the generics just aren't convincing enough or have well known bugs or quirks with texture placements. The idea being to spread the textures across buildings over a moderate area so texture loading times are not noticed and the models use a minimal number over the entire scene. Thus the buildings are rarely at the same elevation.

    The texture consolidation feature in MCX leaves more black space than I prefer, so I tend to create my own textures that don't have as much unused space, and let MCX optimise the textures that don't lend themselves to being done by me manually. This is either through difficulty, or when I get to a point where I've been doing it too long, and I give up my OCD and let MCX sort out the leftovers (usually repeating textures and solid colours).

    And yes, the request was as a more official request based on a sub-discussion buried in another thread.

    I only attached the DAE files as the textures themselves were not important when comparing the XML in the DAE. I can give Arno the textures privately if he wants them, but the DAE files alone were sufficient to highlight the differences and potential ways of detecting individual elements of a DAE file. The zipped or KMZ exports with textures are too large to attach here in their unoptimised (WIP) form, and the last thing I want is someone ripping them to shreds when they aren't complete.

    Sketchup is 2017.

    cheers

    Braedon
     
  14. =rk=

    =rk=

    Joined:
    28 Nov 2009
    Messages:
    1,609
    Country:
    us-washington
    I made a few captures that put some perspective on this request. Below I have a group of three models; a soccer player, a soccer ball and a soccer player in action.

    [​IMG]

    I know I could export each element and its positional data in relation to the whole will be preserved, but this is too much work and the intention is to export the entire model collection and ask MCX to find these divisions. How will it do so? If I do any additional work to the model to define these divisions, should I not just compile each individual element?

    Here is another model. The shrouds have been excluded to conserve polygon complexity. What parameter will MCX measure in order to adequately identify this as a single model?

    [​IMG]

    Altogether, your request is not entirely clear to me. In the first post, you show a tableaux of buildings and express a desire to compile these together but have them broken into individual models.
    Your expressed reason is to simplify texture management.
    Later you write that these building models are to be placed in large urban areas at different elevations.
    Many questions arise from this plan. Surely you can not process all of these buildings; generic, agn, custom, through the same pass of MCX, some sort of standardization will have to be defined and applied, or abandoned. So the question arises, why the small group of buildings? It still seems patently simpler to design and build each model element at a time. Yet you as an artist have created what appears to be 6 buildings and now you want MCX to divide them. Did a different artist create these models? If that were to be the case, then it would sound to me like you want Arno to create a universal translator for models.
     
  15. arno

    arno Administrator Staff Member FSDevConf team Resource contributor

    Joined:
    28 May 2004
    Messages:
    24,831
    Country:
    netherlands
    Supporting drawcall minimizer for multiple objects in a library is on the wishlist already. So that would solve part of this problem.

    I can also see a purpose in splitting on object into different objects on import. If from KMZ keeping their geo location. That might be a useful addition to MCX that can be used in different situations.
     
  16. =rk=

    =rk=

    Joined:
    28 Nov 2009
    Messages:
    1,609
    Country:
    us-washington
    I do not believe it is possible to develop an algorithm that covers all possible variations of the above examples without Watson level software that will, inevitably, inform the artist that he himself is wrong about what defines one portion of art from another. I am avidly watching to be proven wrong.
     
  17. tgibson

    tgibson

    Joined:
    22 Sep 2006
    Messages:
    6,959
    Country:
    us-california
    For Rick's situations I agree, but for a group of buildings it is rather obvious how to split them, and that is by building. Any objects that are (for example) within 0.05" of each other are considered a single object, otherwise they are split.

    And MCX could include a variable distance entry option, where you could specify that distance and thus even Rick's objects could be split properly.
     
  18. =rk=

    =rk=

    Joined:
    28 Nov 2009
    Messages:
    1,609
    Country:
    us-washington
    To be clear, there are no buildings in any models. There are clouds of polygons that when rendered, human eyes interpret to be solid objects. You are asking an analog device to parse these clouds, without using a cursor and human interface to guide it and then throw a virtual lasso around a particular grouping of these clouds and define that as a distinct model. With a SU model, you could select a group and be ready to go, sort of. Except, maybe you picked the group that is all chimneys textured with brick.dds - guess you'd have to pick a different group, the "whole single building" group. I am not saying it can't be done, I have no idea what Arno is capable of, not even sure how I'd use it, but it seems like it would be extremely elegant. It seems like he would want to have a .skp importer and it seems like he would want something like Watson to scan the list of group names for clues as to which included whole, distinct, or whatever paring knife term one would use for calved models.

    Consider the idea of applying a "variable distance" on buildings that may very well be taller than they are removed from each other. In the example of the first picture, it does not appear you could draw a sphere around any one building without intersecting another. So the variable distance filter would have to have a filter or two of it's own as well.
    Then there is the matter of Maya generated models and those of similar origin that actually have no connected faces. If you zoom to an intersection, you will see that polygons have been placed in loose association to each other, occasionally the faces do actually intersect and overlap, with redundant extra polygons created and these models only give the appearance of a continuous construction of connected polygons.
     
  19. tgibson

    tgibson

    Joined:
    22 Sep 2006
    Messages:
    6,959
    Country:
    us-california
    I agree it would be a difficult algorithm to develop, and I don't know if I'd ever use it either...
     
  20. arno

    arno Administrator Staff Member FSDevConf team Resource contributor

    Joined:
    28 May 2004
    Messages:
    24,831
    Country:
    netherlands
    Hi,

    It took a while before I could look at the sample model.

    Logic to detect which polygons form one building are too complex in my opinion, there are too many cases where that might fail.

    But it would be possible to split the object into the different nodes that are stored under the main SketchUp node. In the model with components you can easily identify the three groups of buildings that way. So that could be a relatively simple importer option.

    When loading a KMZ the placement of the model would also have to be updated when splitting I guess.