CEMETECH
Leading The Way To The Future
Login [Register]
Username:
Password:
Autologin:

Don't have an account? Register now to chat, post, use our tools, and much more.
Latest Headlines
Online Users
There are 97 users online: 2 members, 78 guests and 17 bots.
Members: CalebHansberry, flyingfisch.
Bots: VoilaBot (1), Spinn3r (1), Magpie Crawler (3), VoilaBot (1), Googlebot (11).
RSS & Social Media
SAX
You must log in to view the SAX chat widget
Author Message
elfprince13


OVER NINE THOUSAND!


Joined: 23 May 2005
Posts: 10232
Location: A galaxy far far away......

Posted: 16 Jan 2010 01:34:04 pm    Post subject:

Here are the preliminary fields of the data structure that will be used to store LDraw models in engine.


Code:

//Random info
filename (string)
author (string)
bfcCertified (boolean)

parent (ptr, possibly null, to another ldraw object)

//Cached vertex buffers for this file and all subfiles
lineRenderCache (F32 array)
triRenderCache (F32 array)
quadRenderCache (F32 array)

//not quite sure how this will be implemented, but it will store the vertex buffer responsible for drawing the stud textures in place of stud primitives
studRenderCache (F32 array)


lineTransforms  // Point3F array of all line commands stored in *this* file
triTransforms // Point3F array of all tri commands stored in *this* file
quadTransforms // Point3F array of all quad commands stored in *this* file
subFileReferences //string array referencing the names of any .mpd/.ldr/.dat subfiles
subFileTransforms //MatrixF array controlling the transform of the equivalently indexed subfile.


and I'd like to make this a subclass of TSShape to make it really easy to replace existing models with LDraw models, and that should take care of holding the node representations (though I'm not yet sure how to handle the possibility of nodes nested within subfiles.....)

There will also be several global variables controlling stud rendering options and possibly other primitive/part-interceptions, and client connections will have "a selection depth" variable to control how recursively deep to select submodels within a model.

Finally, we'll have some sort of part cache, maintained separately from the models, to prevent commonly used pieces from having to be reparsed over and over.
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative


Back to top
KermMartian


Site Admin


Joined: 14 Mar 2005
Posts: 55741
Location: Earth, Sol, Milky Way

Posted: 17 Jan 2010 12:25:31 am    Post subject:

Dug for parts cache. In all seriousness, though, very nicely done, I of course cannot wait for the final product. Smile
_________________


Back to top
elfprince13


OVER NINE THOUSAND!


Joined: 23 May 2005
Posts: 10232
Location: A galaxy far far away......

Posted: 17 Jan 2010 10:59:15 am    Post subject:

you'll notice that there are actually several different caches Smile

one for raw parts data being read from disk and parsed into memory, and one (three actually) for storing vertex buffers for top level rendered objects, to prevent having to recurse down the entire part tree every time the model is rendered, and to allow the entire model, no matter how many poly's, to be rendered in a fixed number of OpenGL calls.
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative


Back to top
elfprince13


OVER NINE THOUSAND!


Joined: 23 May 2005
Posts: 10232
Location: A galaxy far far away......

Posted: 19 Jan 2010 04:01:30 pm    Post subject:

Thus far today I have fixed FreeBuild so it asks the OS for the proper directory to write log files too. Real handling of this has only been implemented on OS X so far, but the skeleton is there for the others, and just needs me to actually fill in the code. This mean console.log is now written to ~/Library/Logs/Freebuild/net.cemetech.freebuild.log


I've also begun adding the ability to find + use the system LDraw repository before I bother doing any more work with lex's buffer handling.
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative


Back to top
elfprince13


OVER NINE THOUSAND!


Joined: 23 May 2005
Posts: 10232
Location: A galaxy far far away......

Posted: 20 Jan 2010 12:12:11 am    Post subject:

Triple posting spree ftw.




I should mention that under Windows this comes with the caveat of only being able to select LDraw directories on the same drive that FreeBuild was launched from.
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative


Back to top
DShiznit


Guru-in-Training


Joined: 14 Jan 2007
Posts: 3682
Location: The 24th Century

Posted: 20 Jan 2010 07:15:15 am    Post subject:

Because you don't know how to use windows...
Back to top
elfprince13


OVER NINE THOUSAND!


Joined: 23 May 2005
Posts: 10232
Location: A galaxy far far away......

Posted: 20 Jan 2010 09:13:40 am    Post subject:

DShiznit wrote:
Because you don't know how to use windows...


Actually, because I'm using the Torque file browser, and unlike intelligent operating systems, Windows treats / as the root of the current drive, not the root of the whole file system, and I haven't spent any time playing in Windows to get Torque to list drives.
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative


Back to top
DShiznit


Guru-in-Training


Joined: 14 Jan 2007
Posts: 3682
Location: The 24th Century

Posted: 20 Jan 2010 12:40:39 pm    Post subject:

Well then that's torque's fault now isn't it? That being said, I have no problem with this restriction. The LDraw Library isn't THAT big, I can have it on multiple drives if I want FreeBuild on multiple drives.
Back to top
elfprince13


OVER NINE THOUSAND!


Joined: 23 May 2005
Posts: 10232
Location: A galaxy far far away......

Posted: 20 Jan 2010 01:10:38 pm    Post subject:

DShiznit wrote:
Well then that's torque's fault now isn't it? That being said, I have no problem with this restriction. The LDraw Library isn't THAT big, I can have it on multiple drives if I want FreeBuild on multiple drives.


I'm sure I could figure it out, but I'm also having trouble thinking of a situation where someone would be really pissed off by this, so fixing it isn't high on my list of priorities. The sweet thing is that if I run FreeBuild from my mac partition, while in Windows, where my LDraw folder is also saved, I don't even have to change the path in the preferences.
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative


Back to top
elfprince13


OVER NINE THOUSAND!


Joined: 23 May 2005
Posts: 10232
Location: A galaxy far far away......

Posted: 25 Jan 2010 11:04:41 pm    Post subject:

Reading up on the Torque Networking Library, and in particular, the fine details of Ghosting, Tickability, and NetEvents.
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative


Back to top
elfprince13


OVER NINE THOUSAND!


Joined: 23 May 2005
Posts: 10232
Location: A galaxy far far away......

Posted: 09 Feb 2010 01:32:57 am    Post subject:

So as far as I can tell, the lexer and parser for loading LDraw models are ready to go, and I just need to set them up to actually DO something with the data they read.
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative


Back to top
elfprince13


OVER NINE THOUSAND!


Joined: 23 May 2005
Posts: 10232
Location: A galaxy far far away......

Posted: 15 Mar 2011 04:07:15 pm    Post subject:

For you-all's edificaton:
http://sfgp.cemetech.net/freebuild/SketchedOutline-LDrawComponents.pdf
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative




Last edited by elfprince13 on 14 Mar 2012 02:48:46 pm; edited 1 time in total
Back to top
elfprince13


OVER NINE THOUSAND!


Joined: 23 May 2005
Posts: 10232
Location: A galaxy far far away......

Posted: 15 Mar 2011 07:34:56 pm    Post subject:

Double Post, this log shows off a couple clear issues with the parser as it stands right now.

My file loading is having difficulty locating stickered bricks in the s/ subdirectory of parts when they are specified by the full path rather than just the filename.

Secondly, the ~4KB car.dat takes at least a minute to load, whereas the ~300KB blockade_runner.mpd loads almost instantaneously. Presumably this is because the mpd loader attempts to cache things and the normal loader does not. If I then try to load the car a second time, after blockade_runner has loaded, it also loads very quickly, but I get a large number of syntax errors. Presumably this means that once the file is cached, the non-MPD loading mode can find the files there, but is having trouble actually loading them (either due to improper caching, or improper cache-loading).

Quote:
==>ldrawDebugCar();
/Users/thomas/LDRAW
Initializing LDraw Subsystem...
/Users/thomas/LDRAW
Including from disk: 4315.dat
Including from disk: stud3.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4disc.dat
Including from disk: 4-4cyli.dat
Including from disk: stud3.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4disc.dat
Including from disk: 4-4cyli.dat
Including from disk: stud3.dat
...
Including from disk: 4-4cyli.dat
Including from disk: 4-4disc.dat
Including from disk: 179.dat
LDraw Write: Part 179 moved to 4073
Including from disk: 4073.dat
Including from disk: stud4.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4cyli.dat
Including from disk: 4-4cyli.dat
Including from disk: ring3.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4disc.dat
Including from disk: 4-4cyli.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4disc.dat
Including from disk: stud.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4cyli.dat
Including from disk: 4-4disc.dat
Including from disk: 3023.dat
Including from disk: stud3.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4disc.dat
Including from disk: 4-4cyli.dat
Including from disk: box5.dat
Including from disk: box5.dat
Including from disk: stud.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4cyli.dat
Including from disk: 4-4disc.dat
Including from disk: stud.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4cyli.dat
Including from disk: 4-4disc.dat
Including from disk: 3023.dat
Including from disk: stud3.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4disc.dat
Including from disk: 4-4cyli.dat
Including from disk: box5.dat
Including from disk: box5.dat
Including from disk: stud.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4cyli.dat
Including from disk: 4-4disc.dat
Including from disk: stud.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4cyli.dat
Including from disk: 4-4disc.dat
Including from disk: 141.dat
LDraw Write: Part 141 moved to 3822
Including from disk: 3822.dat
Including from disk: s\3822s01.dat
File 's\3822s01.dat' not found in LDraw directories (We check 1 level of primitives, 9 levels of parts, 81 levels of models)
Unable to include file s\3822s01.dat, because it could not be found
Cannot find s\3822s01.dat, ignoring.
...
==>ldrawDebugBR();
/Users/thomas/LDRAW
Initializing LDraw Subsystem...
/Users/thomas/LDRAW
Cached lego10019a3s2r.ldr, used 118 tokens
caching MPD file lego10019a3s3.ldr

Caching MPD subfile lego10019a3s3.ldr
Cached lego10019a3s3.ldr, used 102 tokens
caching MPD file lego10019b.ldr

Caching MPD subfile lego10019b.ldr
Cached lego10019b.ldr, used 5505 tokens
...
==>ldrawDebugCar();
/Users/thomas/LDRAW
Initializing LDraw Subsystem...
/Users/thomas/LDRAW
Including from cache: 4315.dat
Syntax error on line "syntax error"
Syntax error on line "syntax error"
Syntax error on line "syntax error"
Including from cache: 4600.dat
Syntax error on line "syntax error"
Syntax error on line "syntax error"
Syntax error on line "syntax error"
Including from cache: 3031.dat
Syntax error on line "syntax error"
Syntax error on line "syntax error"
Syntax error on line "syntax error"
...


[edit]
Aha, diffing our parsing against the ldlite source shows that I commented out some stuff related to ftype when I rewrote the file finding. The code is set to only cache files that are are of ftype P or PART, not MODEL or OTHER. I think this is probably a big part of it. I've probably mentioned this before, but SourceGear's DiffMerge graphical diff/merge util == <3

[edit 2]
Fixed that, and and a reversed dStricmp (forgot my ! to test for equality rather than inequality). I'm not 100% MPDs are working correctly though, since I'm not getting the profiling dump at the end telling me the cache statistics, but I am for the non-mpd files.
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative


Back to top
elfprince13


OVER NINE THOUSAND!


Joined: 23 May 2005
Posts: 10232
Location: A galaxy far far away......

Posted: 16 Mar 2011 09:16:25 pm    Post subject:

I've implemented the radix trees that will be an integral part of the in-memory cache. Before I continue with that, I need to rework the way the lexer handles an mpd document, because as of right now it caches the whole token stream without executing any of the commands within the file. Non-mpd files appear to be working fine.

[edit]
hextuple post ++
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative


Back to top
CyberPrime


Power User


Joined: 19 Apr 2009
Posts: 370

Posted: 19 Mar 2011 12:19:28 pm    Post subject:

CCCOMBO BREAKER

Anyway, very cool stuff elf. I can't wait to see some in-game renders that look decent Smile
Back to top
elfprince13


OVER NINE THOUSAND!


Joined: 23 May 2005
Posts: 10232
Location: A galaxy far far away......

Posted: 19 Mar 2011 06:38:09 pm    Post subject:

CyberPrime wrote:
CCCOMBO BREAKER

Anyway, very cool stuff elf. I can't wait to see some in-game renders that look decent Smile


Me either! =) I just observed some nice features of the mpd language extensions for LDraw that make it easier to fix the current issues I'm having with the parser.
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative


Back to top
elfprince13


OVER NINE THOUSAND!


Joined: 23 May 2005
Posts: 10232
Location: A galaxy far far away......

Posted: 28 Mar 2011 06:16:41 pm    Post subject:

Since LDraw bricks are not textured in the usual sense of needing images I should be able to do some extreme texture atlasing and cram up to 65k different 4*4 color patches into a single 1024*1024 texture. This should be more than sufficient as the total number of colors that are allowed (by the language specification) at any one time is 512. However, since each file can have its own color definitions that do not scope outside that file, it is important to leave room for models to define their own colors without overriding the default definitions. I don't expect most models to define more than a handful of custom colors, but with this scheme we could have 127 models define the maximum number of colors, and leave the default color table unpolluted.

I will need a few shaders to handle the different material properties, but this number will still be very low.
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative


Back to top
TheStorm


NOU!


Joined: 26 Mar 2007
Posts: 2375

Posted: 10 Apr 2011 11:08:02 pm    Post subject:

Woo, and with s3tc compression you can make it so that the color map texture doesn't need much vram to use and from what I hear it is rather easy to map the 4*4 patches to the face they need to be on.

A thought I just had for the studs would be to maybe use a tiled texture of it with an alpha layer on the top of the bricks so that you only need the color map texture and the stud texture to handle the entire brick. Though I'm sure you already have already thought this part through, so ignore me if I am totally off base.
_________________

"Always code as if the person who will maintain your code is a maniac serial killer that knows where you live" -Unknown

"If you've done something right no one will know that you've done anything at all" -Futurama

"Have a nice day, or not, the choice is yours." Tom Steiner

<Michael_V> or create a Borg collective and call it The 83+
<Michael_V> Lower your slide cases and prepare to be silent linked. Memory clears are futile.
Back to top
elfprince13


OVER NINE THOUSAND!


Joined: 23 May 2005
Posts: 10232
Location: A galaxy far far away......

Posted: 10 Apr 2011 11:31:14 pm    Post subject:

TheStorm wrote:
Woo, and with s3tc compression you can make it so that the color map texture doesn't need much vram to use and from what I hear it is rather easy to map the 4*4 patches to the face they need to be on.

Agreed, although I'm considering upping it to 8*8 depending on the requirements of the Torque material system with regards to UV coordinates for normal maps (for material texturing).

TheStorm wrote:
A thought I just had for the studs would be to maybe use a tiled texture of it with an alpha layer on the top of the bricks so that you only need the color map texture and the stud texture to handle the entire brick. Though I'm sure you already have already thought this part through, so ignore me if I am totally off base.

My tricks with the primitives will probably need to be a little bit more sophisticated, but that sounds okay. I'll probably end up making use of some LOD tricks.
_________________
StickFigure Graphic Productions || VSHI: Vermont Sustainable Heating Initiative


Back to top
KermMartian


Site Admin


Joined: 14 Mar 2005
Posts: 55741
Location: Earth, Sol, Milky Way

Posted: 11 Apr 2011 09:53:08 pm    Post subject:

Whoa, I had missed a rare post from CyberPrime! Great to hear about these efficiency ideas; efficiency will definitely be excellent for those of our members with weaker computers, now that we're going with the more intensive T3D engine.
_________________


Back to top
Display posts from previous:   
Register to Join the Conversation
Have your own thoughts to add to this or any other topic? Want to ask a question, offer a suggestion, share your own programs and projects, upload a file to the file archives, get help with calculator and computer programming, or simply chat with like-minded coders and tech and calculator enthusiasts via the site-wide AJAX SAX widget? Registration for a free Cemetech account only takes a minute.

» Go to Registration page
    » Goto page Previous  1, 2, 3, 4  Next
» View previous topic :: View next topic  
Page 3 of 4 » All times are GMT - 5 Hours

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

© Copyright 2000-2013 Cemetech & Kerm Martian :: Page Execution Time: 0.043531 seconds.