Bi-directional Coupling of Simulation Tools with a Walkthrough System
Bi-directionalCouplingofSimulationTools withaWalkthroughSystem BengtMueck 1,2 HeinzNixdorfInstitute University oPaderborn f [email protected]
WilhelmDangelmaier 1 HeinzNixdorfInstitute University oPaderborn f [email protected]
MatthiasFischer 3,4 HeinzNixdorfInstitute University oPaderborn f [email protected]
WolframKlemisch 3 HeinzNixdorfInstitute University oPaderborn f [email protected] Summary
Visualising im as ethod usedthoelpexperiencing a ndunderstanding causalcohesionsinsimulation processes.Forthispurpose,toolsforvisualising arealreadyimplementedinprevalentsimulation systems.Theusercreateshissimulationmodeland generatesa3-dimensional(2,5-dimensional) visualisingbymeansotfhesimulationsystem.This helpsexaminingtheprocesswhichmakesit easierfortheviewerto“understand”it.Simulatio ntoolsusuallyonlyprovidetheopportunityfora unidirectionalvisualising.Ina3-dimensionalsurr oundingtheviewercannotimplementan interactionwiththesimulationwhilethesystemis running.Thoughaninteractionduringthe simulationrunenablestheusertogainabetterun derstandingofcausalcohesions.Solutionsvia HLA are sophisticatedand therefore rather suitedf or extensive projects. Wepresentadistributedsystemconsistingofacom mercialmanufacturingsimulationtool,a couplingmoduleandawalkthroughsystem.Thedistr ibutedsysteminconjunctionswiththe couplingmoduleguarantees generalityandwide a fieldoapplications f othe f walkthroughsystem. Furtheritguaranteesflexibilityandselectionof thespecializedgraphicshardwareforthe walkthroughsystem.Afurthercontributionothis f paperisthesolutionothe f timesynchronisation problem causedbsyimulationtoolandwalkthroughs ystem.
1Introduction 1.1Problem Avisualisingofsimulationprocessesfortheviewe experienceosimulation f processes.Aninteraction theviewermoredirectlyintotheprocessandthus 1 Heinz Nixdorf Institute,Business Computing,esp. 2 Partially supported bD y FG grantDA 155/24-2 3 Heinz Nixdorf Institute,Theoretical Computer Scie 4 PartiallysupportedbyDFGgrantME872/8-1andth EU under contract numberIST-1999-14186 (ALCOM-FT)
rfacilitateshisunderstandingand betweenhimandthesysteminvolves providesforabetterunderstanding. CIM,D-39095Paderborn,Germany nce,D-39095Paderborn,Germany F e utureandEmergingTechnologiesprogramme
ofthe
Mostcommercialmanufacturingsimulationtools[KlG 99]allowforavisualisingotfhe runbutnotforaninteraction.Thecallforanint eractionduringthesimulationprocessof aconventionalmanufacturingsimulatorcausesmany problemswhichwillbediscussed in thispaper.Furthermore,wedevelop solution a a rchitecture,implementitandintroduce theresults.
1.2DemandsontheSolution Togenerate no proprietaryspecialisedsolutionsthesystemshould beasflexibleas possible,thus itshouldbpeossibletoemploy vari ous simulation tools. Beforethesimulationiscarriedoutamodelhasto becreatedwhichusuallydoesnot containanyinformationonthevisualising.Inorde to rguaranteeanefficienthandlingof theentiresystemitisfundamentaltogenerateav isualisingmodeloutofthesimulation model.Duringthecreationofthesimulationmodel itshouldbefeasibletoimplicitly modelpositions andmovements of theobjects in the visualising.
1.3TimeSynchronisationProblem Ifsimulationandwalkthroughsystemrunindependen tlytheybothoperatewithintheir owntimescheme.Thisimpliescertainrequirements: Eventsinthesimulationmay triggervisualisingevents.Thosecannotbepictur edaccurately,iftheyarehappeningin thepastotfhewalkthroughsystem-time.Consequent ly,walkthroughsystem-timemay neveroutrunsimulationtime.Ifinteractionsaret riggeredinawalkthroughsystem,itis onlypossibletotransferthemintothesimulation atthecurrentpointofsimulationtime withouthaving toresetsimulation.Theviewerhastowaitadequat elylongforraeaction inthevisualising,ifsimulationtimeisignifica ntlyfasterthanwalkthroughsystemtime. Hence,simulationtimemaynotbeconsiderablyfast erthanwalkthroughsystemtimein ordertoguaranteethefeelingofadirectreaction We . willdealwithtwoproceduresto solvethisproblem.Oneotfhosehasbeenimplement edandexaminedinourprototypic system.
1.4RelatedWork Thecouplingofsimulatorsandvisualisingsystems viaHLA[DMS99][Jes01]both pursuesimilaraims.HLAoffersanextensiverange offunctionsbutisrather sophisticatedinitsuse.Withonlyfewexceptions [LSU99] HLAisnotwidelyaccepted intheworldofproductionsimulators[KlGu99].The conceptthatwearediscussingin thispaperislesscomprehensive,smallerandspeci alisedonthebi-directionalcouplingof simulatorswithinwalkthroughsystems. Timesynchronisationproblemsareocf entralimport ancefortheimplementationof3dimensionalMultiplayer-Games.ExamplesareDoom,D ukeNukem,UltimaOnline, Unrealortheego-shooterQuake [Abr97].Inthebe ginningthesimulationingameslike Doom or DukeNukemwas carriedoutstrictlysynchro nous(peer-to-peer)[Swe99].Later ontheneedforabroaderscalabilitycausedatran sitiontoclient-servertechnologyin gameslikeUltimaOnlineandQuake[Abr00].Thegam e,includingitscurrentplayer positionsandactions,is imulatedonacentralse rverwhilethesimulationdatesare broadcastedtotheclients.In later versions of Quakesimulation andpredictionl ogicwere
usedontheclientside[Swe99],whichaccountedfo rinconsistenciesbetweenthe predictionoftheclientandthesimulationofthe server.Topreventthiskindof inconsistencieswde idnotintegratepredictionand simulationlogicintoourwalkthrough system.Thisguaranteesawalkthroughsystemwhich isratheruniversalandnot dependenton thedeployedsimulator.
2Architecture Theentiresystemprovidestwofocalfunctionalitie tobeabletocreatemodels.Ontheotherhand,he interactively.Thereforewsehouldspeak otwo f ind user level
model level
fsortheuser.Ontheonehand,hehas needsto“experience”thesimulation ependentusers (modeller andviewer): modeller
simulation model
simulation tool
viewer
3D representatives
coupling
software level standardcomponents
walkthrough system new components
Figure 1:
Architecture othe f entire system with view a onea
chlevel
Theviewer (seefig.1) experiencesandinteractswiththesystemviaavis ualising component.Hedoesnothaveaccesstoanyotheruse irnterface.Themodellerutilizes toolstocreatemodelsforthesimulationaw s ella fsorthe3-dimensionalvisualising.In ourcurrentconcepthewillgeneratesimulationmod elsinasimulationsurrounding.For thevisualisingsimulationobjectsareattributedt o3-dimensionalrepresentatives(seefig. 1).Thoserepresentativeswillbecreatedbyconven tionalmodellingtoolsfor3dimensionalmodels.Simulationobjectsandtheir3dimensionalrepresentativesare strictly separated. Thevisualisingsystem consists of twoadditionals oftwarecomponents, whicharethecouplingmoduleontheonehandandt hewalkthroughsystemontheother hand(see fig.1).Bothcomponentsrepresentindividualproce sses. Asindividualmodules may communicateviaTCP/IPthey can biemplemented on various computers. Throughastrict modularisationandacommunicationamongtheindivi dualpartswhich embracesallcomputers,diversehardwarecanbeemp loyed.Forinstance,this architectureallowsfortheimplementationofspeci alisedvisualisinghardware.Inthe followingww e illdiscuss thetasks andfunctionali ties of each component.
2.1TasksoftheSimulationTools Thesimulationtoolconductsthedynamiccalculatio nswhichhavebeenspecified throughthesimulationmodel.Actions(e.g.movemen tsofobjects)withinthe walkthroughsystemarestimulatedbythoseinstrume nts.Ontheotherhand,the simulationtoolhastointegrateuserinteractions ofthewalkthroughsystemintothe
simulationwithinrunningtime. Thesimulationtooldoesnotonlysupporttheexecu butalsothecreationosfimulationmodels.Inthis modelweconsiderthe3-dimensional compositionofobjectswhilemodellinginorderto enablefastandeasyworkwiththe system(themodelshouldatleastcontaintheposit ionofobjectsononeplane).We thereforecurrently employ 2-dimensionalcoordinate of sthesimulationobjectswithinthe simulationtools.The3-dimensionalsceneicsreate dbythecouplingmoduleandthe3dimensionalrepresentativeswhichhavebeen produce dbefore.
tion
2.2TasksoftheWalkthroughSystem Thewalkthroughsystemrepresentstheinterfacebet viewer.Ivt isualisesthesimulationfortheviewer interactionhas twoimplicationsfor thewalkthroug • •
weentheentiresystemandthe andpicksuphisinteractions. shystem:
The
Navigationthroughthescene: Theviewershouldbeabletomovefreely withinthevisualisinginthesimulation,whichmea nshsehouldbaebletomove towards various places andturn andmovethewhole scene. Actionsforsimulationobjects: Theviewershouldbeabletoinfluencethe simulationprocedurebyanaimed“picking”osimul f ationobjects.Itshould,for example,bepossibletomoveaswitchonaconveyor belt.Thewalkthrough systemreportstothecouplingmodulewhichaction orobjecttheviewerhas chosen.Thecouplingmodulethen decideswhich acti ons totake.
2.3TasksoftheCouplingModule Thecouplingmoduleprovidesfortheconnectinginf rastructureotfheindividualparts. Allcommunicationbetweenthewalkthroughsysteman dthesimulationtoolwillbe modularisedhere.Itcapsulesspecificcommunicatio ninterfacesbetweenthesimulation toolsandthewalkthroughsystemandfurthermoreun dertakesabi-directional transformationom f essagesintotheappropriatelan guage. Neitherthesimulationsystem northewalkthroughsystemhavetobaelteredfort herespectivepartner.Itispossibleto simplyconnectandtriggereventsfromoutofthes imulation.Thecapsulingofthe couplingmoduleseizesspecificcharacteristicsof thesimulationaswellasotfheused simulation tools: •
Simulation tool:inthecouplingmodulespecificcharacteristicsof theprotocol andtheparametersofdiversesimulationtoolscan behiddenfromthe walkthroughsystem andpicturedoanunniversalint erface. • Simulation:simulation-specificcharacteristicslike,forexam ple,theshiftofa switch require feedback a tothesimulation tool.T hedecision ian fobject should deliverfaeedbackonot r shouldbteakenwithoutc onsideringthepickingothe f objectin thewalkthrough system within thecouplin gmodule. This willresultinastrictseparationofunctionaliti eswithinthesimulationsystemand thewalkthrough system andwillmakethewalkthroug shystem universally applicable. Aninterfacewhichhasbeenformallydefinedineac hdirectionenablestheexchangeof Openinterfacesenablenotonlythe bothsimulationtoolsandthewalkthroughsystem.
implementationofvarioussimulationtoolsbutalso visualisingtools.Inthecurrentrealisationevery individualcommunicationchanneltothecouplingmo existsonlyonechannelfromthecouplingmoduleto couplingmoduleensures caorrectattribution ome f
theimplementationofdiverse simulationobjectestablishesan dule.Asopposedtothis,there thewalkthroughsystem.The ssages totheobjects.
3Modelling 3.1Modelling withintheSimulation Thesimulationtoolsshouldnotonlydeterminethe shouldalsodefinethepositionopf ermanentobject descriptionsshouldbaevailablei3-dimensional na identifiedthrough explicitidentifierswithin the Theemployedsimulationtool(currentlyEM-Plant) coordinatesonthisplainhelpcalculatingthecoor visualising.Valuableandintuitivemodelscanbec dimensionalobjectstoitsarrangement-plainisequ dimensionalobjectstoitssurface.Tovisualisefu directlyinvolvedinthesimulation(e.g.thefloor created,thatcan bveisualisedbuthavenoinfluen
modellingosfimulationmodels,but w s ithinthevisualisation.Therefore format.Thosedescriptionsneedtobe simulation. arrangestheobjectsonaplain.The dinatesfora21/2dimensional reated,iftheproportionofthe2altotheproportionofthe3rtherstaticobjects,whicharenot w , alls,workeretc.)typeshavetobe ceotnheprocedureothe f simulation.
3.2ModellingwithintheWalkthroughSystem Theconfigurationostatic f objectswillbeperform edads escribedabove.Thepositionof moveableobjectswillbecalculatedduringtherunn ingtimeothe f simulation.Acertain amountof3-dimensionalrepresentativeshavetobe retainedfortheobjects.Thosehave tobeformedwith thehelpofconventionalmodellingtools(3dStudio Max,Softimage etc.)andshouldbceopiedfrom thebasis intothe walkthroughsystem ifnecessary.
3.3ConstructionofaStaticScene Totransmitallstaticvisualisingdatestothewal initialisingeventwillbesendfromthecouplingm simulation.Eachstaticobjectinthesimulationth transmitshiscoordinatesincombinationwithhist dimensionalrepresentative)tothewalkthroughsyst thesedates toestablish static a representation of modeller
kthroughsystemafterthemodellingan oduletoallobjectswithinthe encalculatesitsownpositionand ype(resp.theidentifierofhis3em.Thewalkthroughsystememploys thescenes.
simulationtool coupling
+
3Drepresentatives
Figure 2: Construction osaftatic scene
static 3Dgeneration
3Dscene walkthroughsystem
Figure2showstheconstructionprocessofastatic dimensionalrepresentatives,whichhadbeendevelop hadbeendeliveredbythecouplingmodule,thewalk dimensionalscene.Thecouplingmodulereceivesthe coordinatesfromthesimulationtool.Thethirdcoo moduleaccordingtothespecificsimulationtoolan executedbeforetheactualsimulationwithinitspr box below:
scene.Withthehelpofthe3edearlier,andtheobjectIDs,which throughsystemcreatesastatic3objectsandits2-dimensional rdinateibseingaddedbythecoupling dthesimulation.Theentireprocessis eprocessingandisummarizedinthe
Steps forprocessing: Staticconstructionothe f scene btyhe modeller 1. Creation osafimulation descriptionwith simulat a 2. Creation o3af-dimensionalrepresentative 3. Simulation-specificadjustmenttothecouplingmodu 4. Construction osaftaticsceneosimulation f tool, walkthroughsystem
4Bi-directionalCoupled
ion tool le couplingmoduleand
DynamicSimulation
Inthismodelweimplementtheloosecouplingofsi mulationandwalkthroughsystem, whilebothsystemshaveseparated instancesoobjects. f Changeswhichhavetobedit inbothsystemssynchronouslyaremainlytheproduc tiononew f objects,thedestruction ofoldobjectsandthemovementofobjectsanduser interactionswhicharedirectedaan t object.Messageshavetobdeefinedaleast t for th eseoperations. viewer
interaction
rendering engine
simulation tool coupling 3Drepresentatives
+
ed
dynamic 3Dgeneration
3Dszene walkthroughsystem
Figure 3: Bi-directional dynamicsimulationprocedure
Thesimulationmodulehastobeabletocreatenew objectswithinthewalkthrough system.Thisnecessitatesthetransmissionofanid entifierofa3-dimensional representative(see fig.3),itspositionandanidentifyforfurtherc ommunicationabout theseobjects from thesimulationmoduletothewal kthroughsystem. Ifobjectsinthewalkthroughsystemaretobedest royed,theidentifierotfhoseobjects hastobteransmittedtothewalkthroughsystem.To moveobjects,starting-time,arrivaltimeanddestination-positioninghavetobbe roadca sted.Itisalsopossibletoconvey the starting-time,thedirection,velocityandthedura tion. Thewalkthroughsystemaccepts userinteractionsandhastoforwardthose(seefig 3). . Thisnecessitatestransmission a of objectIDandthetypeoftheinteractiontothesim ulation. Furthertransmitted functionalities likee.g.information aboutthesyn chronisation otime f arepossible.
Message CreateObject DestroyObject MoveObject UserInteraction
Direction Sim. ->Walk. Sim. ->Walk. Sim. ->Walk. Walk. ->Sim.
Attributes ObjectID,ID 3DRepresentative,Position ObjectID ObjectID,Startingtime,Destination-time, Destination-position ObjectID, TypeoInteraction f
Table 1: Examplesof some objects
4.1CouplingModule Thecouplingmodulecollectsmessages,whichstemf romtheconnectionstotheobjects ofthesimulation,andforwardsthosetothewalkth roughsystem.Furthermore,the objectIDhelpsdistributingthemessagesaboutuser interactions,whichcomeoutofthe walkthroughsystem,throughtheadequatechannelst othesimulator.Ourgoalforthe couplingmoduleistocapsulateallsimulationand simulationtoolsdependenttasks. Typicalimplementedtasks ofour system are: • Transfer of coordinates. • Storageoftheobjectcoordinates,e.g.duringmove mentofanobjectfrom sourcetodestination. • Storageothe f objectsizefor raightplacing itn hescene. • Initialisation,creation,anddeletionofmovingob jects.Thesimulationonly ceratethestartandendtimeothe f object. • Computation othe f speedomoving f objects. AllthecommunicationwillgothroughaTCP/IPconn ection(stream),whichhasbeen establishedonceinthebeginningothe f simulation Each . package whichhasbeensendis equivalentto produced a message(e.g.“createobje ct”o“move r object”).Itis 96bytebig andalwayscontainsthedata-fieldstypeom f essage o, bjectIDandstarting-timeotfhe objectinthesimulation.DependingontheobjectID data-fieldslikeexpiring-date, startingposition,endpositionorotation r etc.ma ybeadded.Forthereceiptathreadhas beenimplementedinthewalkthroughsystemwhichis runningparalleltothethreadfor therendering othe f walkthroughsystem.
4.2TimeSynchronisation Inconcretesimulationssimulationtimejumpsfrom other.Calculationshappenaone t pointofsimulati thesimulationtimeisrequiredtomoveforwardin simulationtimeprogressesintotalhavetobecalc withinthesimulatorshastobestoppedregularly. simulationtimeinordertopreventinconsistencies simulators.Tocompensatefortheuseocalculatio f forerun.Thereforeprogressesinthesimulationcan time.Alargeforerunallowsforthecompensationo load,butpreventsafastuserreactiontoevents. differentuserscenarios.Itwouldbeadvantageous
onepointofsimulationtimetothe ontime,thoughtheyneedrealtime.If (resp.proportionalto)realtime, ulatedfasterthanrealtime.Time Visualisingtimeshouldnotoutrun betweenvisualisingandthe ntime,simulatorshavetobe givena bevisualisedinrealtime=simulation bf iggervariationsin computation Varyingfastreactionsaresensiblein tokeepforeruntimeforthese
experimentsvariable.Ifvisualisingtimeapproache ssimulationtimecrucially,a deceleration rather than halt a of thevisualising helps theviewer toget caompletepicture ofthesystem.Visualisationsarecalculatingduet otherhythmoftheset-upofnew pictures.Thepicturerepetitionrateothe f visual isingshouldbde ecoupledfromthetime progress of simulation. standsforthesimulationtimeof Let t betherealtimeoof ursystem.Furthermore ts(t) oolsthere thesimulatorand tw(t)forthewalkthroughsystemtime.Withsimulationt isnotlinearto t, whichmeansthatit oftenoccurstheproblemthatsimulationtime ts(t) doesnapply ot toalltimes t:t=t s(t). Inmanysimulationtoolswalkthroughsystemtime runs in partly a constant,stair-likefunction (seefig. 4). ts(t),t w(t)
t ts(t)
event k event j event i
} computation timefor event
j t
Figure 4:
Realtime andsimulationtime
Thesimulatorwillalwaystrytokeepclosetoreal time t. Thisproblemwillintensifyina systemwithwalkthrough a systemwhichibsi-direct ionallycoupled,astheentiresystem tw(t). This doesnotonlyincludesimulationtime ts(t) butalsowalkthroughsystemtime may causeproblems becauseovarious f reasons: • Delaythroughtransmission: Awalkthroughsystemwhichisrunningon anothercomputercauseshightransmissiontimes.Th esemayresultindelayson both t computers,whensimulationstatusointerac r tionspecificationsothe f user aresentfrom onecomputer totheother. • Divergingotfimebases: Therelativetimebasisothe f hardwaretimerinbo th computersmaydivergeinthecourseotime f andthe reforecomplicateanexact synchronisation. Thismeansthatitwillgetveryhardtoguarantee for alltimes: t:t w(t)=t s(t).Theterm sbehindthe follow-uptime ∆tswstandsforthetimewhichthewalkthroughtimelag the time which the walkthrough simulationtime.Correspondingly, ∆trepresents w system lags behindtherealtime: = ts(t) t- w(t) ∆tw(t) = t t- w(t) ∆tsw(t) Follow-uptimes causeproblemsfortheuseothe f simulator,ifthe simulationprocedure ischangingduetoaspecificationoftheviewerin thewalkthroughsystem.Besides,it identified: may getproblematicithe f progression osimulatio f tnime ts(t) isnotclearly tw(t)
WilhelmDangelmaier 1 HeinzNixdorfInstitute University oPaderborn f [email protected]
MatthiasFischer 3,4 HeinzNixdorfInstitute University oPaderborn f [email protected]
WolframKlemisch 3 HeinzNixdorfInstitute University oPaderborn f [email protected] Summary
Visualising im as ethod usedthoelpexperiencing a ndunderstanding causalcohesionsinsimulation processes.Forthispurpose,toolsforvisualising arealreadyimplementedinprevalentsimulation systems.Theusercreateshissimulationmodeland generatesa3-dimensional(2,5-dimensional) visualisingbymeansotfhesimulationsystem.This helpsexaminingtheprocesswhichmakesit easierfortheviewerto“understand”it.Simulatio ntoolsusuallyonlyprovidetheopportunityfora unidirectionalvisualising.Ina3-dimensionalsurr oundingtheviewercannotimplementan interactionwiththesimulationwhilethesystemis running.Thoughaninteractionduringthe simulationrunenablestheusertogainabetterun derstandingofcausalcohesions.Solutionsvia HLA are sophisticatedand therefore rather suitedf or extensive projects. Wepresentadistributedsystemconsistingofacom mercialmanufacturingsimulationtool,a couplingmoduleandawalkthroughsystem.Thedistr ibutedsysteminconjunctionswiththe couplingmoduleguarantees generalityandwide a fieldoapplications f othe f walkthroughsystem. Furtheritguaranteesflexibilityandselectionof thespecializedgraphicshardwareforthe walkthroughsystem.Afurthercontributionothis f paperisthesolutionothe f timesynchronisation problem causedbsyimulationtoolandwalkthroughs ystem.
1Introduction 1.1Problem Avisualisingofsimulationprocessesfortheviewe experienceosimulation f processes.Aninteraction theviewermoredirectlyintotheprocessandthus 1 Heinz Nixdorf Institute,Business Computing,esp. 2 Partially supported bD y FG grantDA 155/24-2 3 Heinz Nixdorf Institute,Theoretical Computer Scie 4 PartiallysupportedbyDFGgrantME872/8-1andth EU under contract numberIST-1999-14186 (ALCOM-FT)
rfacilitateshisunderstandingand betweenhimandthesysteminvolves providesforabetterunderstanding. CIM,D-39095Paderborn,Germany nce,D-39095Paderborn,Germany F e utureandEmergingTechnologiesprogramme
ofthe
Mostcommercialmanufacturingsimulationtools[KlG 99]allowforavisualisingotfhe runbutnotforaninteraction.Thecallforanint eractionduringthesimulationprocessof aconventionalmanufacturingsimulatorcausesmany problemswhichwillbediscussed in thispaper.Furthermore,wedevelop solution a a rchitecture,implementitandintroduce theresults.
1.2DemandsontheSolution Togenerate no proprietaryspecialisedsolutionsthesystemshould beasflexibleas possible,thus itshouldbpeossibletoemploy vari ous simulation tools. Beforethesimulationiscarriedoutamodelhasto becreatedwhichusuallydoesnot containanyinformationonthevisualising.Inorde to rguaranteeanefficienthandlingof theentiresystemitisfundamentaltogenerateav isualisingmodeloutofthesimulation model.Duringthecreationofthesimulationmodel itshouldbefeasibletoimplicitly modelpositions andmovements of theobjects in the visualising.
1.3TimeSynchronisationProblem Ifsimulationandwalkthroughsystemrunindependen tlytheybothoperatewithintheir owntimescheme.Thisimpliescertainrequirements: Eventsinthesimulationmay triggervisualisingevents.Thosecannotbepictur edaccurately,iftheyarehappeningin thepastotfhewalkthroughsystem-time.Consequent ly,walkthroughsystem-timemay neveroutrunsimulationtime.Ifinteractionsaret riggeredinawalkthroughsystem,itis onlypossibletotransferthemintothesimulation atthecurrentpointofsimulationtime withouthaving toresetsimulation.Theviewerhastowaitadequat elylongforraeaction inthevisualising,ifsimulationtimeisignifica ntlyfasterthanwalkthroughsystemtime. Hence,simulationtimemaynotbeconsiderablyfast erthanwalkthroughsystemtimein ordertoguaranteethefeelingofadirectreaction We . willdealwithtwoproceduresto solvethisproblem.Oneotfhosehasbeenimplement edandexaminedinourprototypic system.
1.4RelatedWork Thecouplingofsimulatorsandvisualisingsystems viaHLA[DMS99][Jes01]both pursuesimilaraims.HLAoffersanextensiverange offunctionsbutisrather sophisticatedinitsuse.Withonlyfewexceptions [LSU99] HLAisnotwidelyaccepted intheworldofproductionsimulators[KlGu99].The conceptthatwearediscussingin thispaperislesscomprehensive,smallerandspeci alisedonthebi-directionalcouplingof simulatorswithinwalkthroughsystems. Timesynchronisationproblemsareocf entralimport ancefortheimplementationof3dimensionalMultiplayer-Games.ExamplesareDoom,D ukeNukem,UltimaOnline, Unrealortheego-shooterQuake [Abr97].Inthebe ginningthesimulationingameslike Doom or DukeNukemwas carriedoutstrictlysynchro nous(peer-to-peer)[Swe99].Later ontheneedforabroaderscalabilitycausedatran sitiontoclient-servertechnologyin gameslikeUltimaOnlineandQuake[Abr00].Thegam e,includingitscurrentplayer positionsandactions,is imulatedonacentralse rverwhilethesimulationdatesare broadcastedtotheclients.In later versions of Quakesimulation andpredictionl ogicwere
usedontheclientside[Swe99],whichaccountedfo rinconsistenciesbetweenthe predictionoftheclientandthesimulationofthe server.Topreventthiskindof inconsistencieswde idnotintegratepredictionand simulationlogicintoourwalkthrough system.Thisguaranteesawalkthroughsystemwhich isratheruniversalandnot dependenton thedeployedsimulator.
2Architecture Theentiresystemprovidestwofocalfunctionalitie tobeabletocreatemodels.Ontheotherhand,he interactively.Thereforewsehouldspeak otwo f ind user level
model level
fsortheuser.Ontheonehand,hehas needsto“experience”thesimulation ependentusers (modeller andviewer): modeller
simulation model
simulation tool
viewer
3D representatives
coupling
software level standardcomponents
walkthrough system new components
Figure 1:
Architecture othe f entire system with view a onea
chlevel
Theviewer (seefig.1) experiencesandinteractswiththesystemviaavis ualising component.Hedoesnothaveaccesstoanyotheruse irnterface.Themodellerutilizes toolstocreatemodelsforthesimulationaw s ella fsorthe3-dimensionalvisualising.In ourcurrentconcepthewillgeneratesimulationmod elsinasimulationsurrounding.For thevisualisingsimulationobjectsareattributedt o3-dimensionalrepresentatives(seefig. 1).Thoserepresentativeswillbecreatedbyconven tionalmodellingtoolsfor3dimensionalmodels.Simulationobjectsandtheir3dimensionalrepresentativesare strictly separated. Thevisualisingsystem consists of twoadditionals oftwarecomponents, whicharethecouplingmoduleontheonehandandt hewalkthroughsystemontheother hand(see fig.1).Bothcomponentsrepresentindividualproce sses. Asindividualmodules may communicateviaTCP/IPthey can biemplemented on various computers. Throughastrict modularisationandacommunicationamongtheindivi dualpartswhich embracesallcomputers,diversehardwarecanbeemp loyed.Forinstance,this architectureallowsfortheimplementationofspeci alisedvisualisinghardware.Inthe followingww e illdiscuss thetasks andfunctionali ties of each component.
2.1TasksoftheSimulationTools Thesimulationtoolconductsthedynamiccalculatio nswhichhavebeenspecified throughthesimulationmodel.Actions(e.g.movemen tsofobjects)withinthe walkthroughsystemarestimulatedbythoseinstrume nts.Ontheotherhand,the simulationtoolhastointegrateuserinteractions ofthewalkthroughsystemintothe
simulationwithinrunningtime. Thesimulationtooldoesnotonlysupporttheexecu butalsothecreationosfimulationmodels.Inthis modelweconsiderthe3-dimensional compositionofobjectswhilemodellinginorderto enablefastandeasyworkwiththe system(themodelshouldatleastcontaintheposit ionofobjectsononeplane).We thereforecurrently employ 2-dimensionalcoordinate of sthesimulationobjectswithinthe simulationtools.The3-dimensionalsceneicsreate dbythecouplingmoduleandthe3dimensionalrepresentativeswhichhavebeen produce dbefore.
tion
2.2TasksoftheWalkthroughSystem Thewalkthroughsystemrepresentstheinterfacebet viewer.Ivt isualisesthesimulationfortheviewer interactionhas twoimplicationsfor thewalkthroug • •
weentheentiresystemandthe andpicksuphisinteractions. shystem:
The
Navigationthroughthescene: Theviewershouldbeabletomovefreely withinthevisualisinginthesimulation,whichmea nshsehouldbaebletomove towards various places andturn andmovethewhole scene. Actionsforsimulationobjects: Theviewershouldbeabletoinfluencethe simulationprocedurebyanaimed“picking”osimul f ationobjects.Itshould,for example,bepossibletomoveaswitchonaconveyor belt.Thewalkthrough systemreportstothecouplingmodulewhichaction orobjecttheviewerhas chosen.Thecouplingmodulethen decideswhich acti ons totake.
2.3TasksoftheCouplingModule Thecouplingmoduleprovidesfortheconnectinginf rastructureotfheindividualparts. Allcommunicationbetweenthewalkthroughsysteman dthesimulationtoolwillbe modularisedhere.Itcapsulesspecificcommunicatio ninterfacesbetweenthesimulation toolsandthewalkthroughsystemandfurthermoreun dertakesabi-directional transformationom f essagesintotheappropriatelan guage. Neitherthesimulationsystem northewalkthroughsystemhavetobaelteredfort herespectivepartner.Itispossibleto simplyconnectandtriggereventsfromoutofthes imulation.Thecapsulingofthe couplingmoduleseizesspecificcharacteristicsof thesimulationaswellasotfheused simulation tools: •
Simulation tool:inthecouplingmodulespecificcharacteristicsof theprotocol andtheparametersofdiversesimulationtoolscan behiddenfromthe walkthroughsystem andpicturedoanunniversalint erface. • Simulation:simulation-specificcharacteristicslike,forexam ple,theshiftofa switch require feedback a tothesimulation tool.T hedecision ian fobject should deliverfaeedbackonot r shouldbteakenwithoutc onsideringthepickingothe f objectin thewalkthrough system within thecouplin gmodule. This willresultinastrictseparationofunctionaliti eswithinthesimulationsystemand thewalkthrough system andwillmakethewalkthroug shystem universally applicable. Aninterfacewhichhasbeenformallydefinedineac hdirectionenablestheexchangeof Openinterfacesenablenotonlythe bothsimulationtoolsandthewalkthroughsystem.
implementationofvarioussimulationtoolsbutalso visualisingtools.Inthecurrentrealisationevery individualcommunicationchanneltothecouplingmo existsonlyonechannelfromthecouplingmoduleto couplingmoduleensures caorrectattribution ome f
theimplementationofdiverse simulationobjectestablishesan dule.Asopposedtothis,there thewalkthroughsystem.The ssages totheobjects.
3Modelling 3.1Modelling withintheSimulation Thesimulationtoolsshouldnotonlydeterminethe shouldalsodefinethepositionopf ermanentobject descriptionsshouldbaevailablei3-dimensional na identifiedthrough explicitidentifierswithin the Theemployedsimulationtool(currentlyEM-Plant) coordinatesonthisplainhelpcalculatingthecoor visualising.Valuableandintuitivemodelscanbec dimensionalobjectstoitsarrangement-plainisequ dimensionalobjectstoitssurface.Tovisualisefu directlyinvolvedinthesimulation(e.g.thefloor created,thatcan bveisualisedbuthavenoinfluen
modellingosfimulationmodels,but w s ithinthevisualisation.Therefore format.Thosedescriptionsneedtobe simulation. arrangestheobjectsonaplain.The dinatesfora21/2dimensional reated,iftheproportionofthe2altotheproportionofthe3rtherstaticobjects,whicharenot w , alls,workeretc.)typeshavetobe ceotnheprocedureothe f simulation.
3.2ModellingwithintheWalkthroughSystem Theconfigurationostatic f objectswillbeperform edads escribedabove.Thepositionof moveableobjectswillbecalculatedduringtherunn ingtimeothe f simulation.Acertain amountof3-dimensionalrepresentativeshavetobe retainedfortheobjects.Thosehave tobeformedwith thehelpofconventionalmodellingtools(3dStudio Max,Softimage etc.)andshouldbceopiedfrom thebasis intothe walkthroughsystem ifnecessary.
3.3ConstructionofaStaticScene Totransmitallstaticvisualisingdatestothewal initialisingeventwillbesendfromthecouplingm simulation.Eachstaticobjectinthesimulationth transmitshiscoordinatesincombinationwithhist dimensionalrepresentative)tothewalkthroughsyst thesedates toestablish static a representation of modeller
kthroughsystemafterthemodellingan oduletoallobjectswithinthe encalculatesitsownpositionand ype(resp.theidentifierofhis3em.Thewalkthroughsystememploys thescenes.
simulationtool coupling
+
3Drepresentatives
Figure 2: Construction osaftatic scene
static 3Dgeneration
3Dscene walkthroughsystem
Figure2showstheconstructionprocessofastatic dimensionalrepresentatives,whichhadbeendevelop hadbeendeliveredbythecouplingmodule,thewalk dimensionalscene.Thecouplingmodulereceivesthe coordinatesfromthesimulationtool.Thethirdcoo moduleaccordingtothespecificsimulationtoolan executedbeforetheactualsimulationwithinitspr box below:
scene.Withthehelpofthe3edearlier,andtheobjectIDs,which throughsystemcreatesastatic3objectsandits2-dimensional rdinateibseingaddedbythecoupling dthesimulation.Theentireprocessis eprocessingandisummarizedinthe
Steps forprocessing: Staticconstructionothe f scene btyhe modeller 1. Creation osafimulation descriptionwith simulat a 2. Creation o3af-dimensionalrepresentative 3. Simulation-specificadjustmenttothecouplingmodu 4. Construction osaftaticsceneosimulation f tool, walkthroughsystem
4Bi-directionalCoupled
ion tool le couplingmoduleand
DynamicSimulation
Inthismodelweimplementtheloosecouplingofsi mulationandwalkthroughsystem, whilebothsystemshaveseparated instancesoobjects. f Changeswhichhavetobedit inbothsystemssynchronouslyaremainlytheproduc tiononew f objects,thedestruction ofoldobjectsandthemovementofobjectsanduser interactionswhicharedirectedaan t object.Messageshavetobdeefinedaleast t for th eseoperations. viewer
interaction
rendering engine
simulation tool coupling 3Drepresentatives
+
ed
dynamic 3Dgeneration
3Dszene walkthroughsystem
Figure 3: Bi-directional dynamicsimulationprocedure
Thesimulationmodulehastobeabletocreatenew objectswithinthewalkthrough system.Thisnecessitatesthetransmissionofanid entifierofa3-dimensional representative(see fig.3),itspositionandanidentifyforfurtherc ommunicationabout theseobjects from thesimulationmoduletothewal kthroughsystem. Ifobjectsinthewalkthroughsystemaretobedest royed,theidentifierotfhoseobjects hastobteransmittedtothewalkthroughsystem.To moveobjects,starting-time,arrivaltimeanddestination-positioninghavetobbe roadca sted.Itisalsopossibletoconvey the starting-time,thedirection,velocityandthedura tion. Thewalkthroughsystemaccepts userinteractionsandhastoforwardthose(seefig 3). . Thisnecessitatestransmission a of objectIDandthetypeoftheinteractiontothesim ulation. Furthertransmitted functionalities likee.g.information aboutthesyn chronisation otime f arepossible.
Message CreateObject DestroyObject MoveObject UserInteraction
Direction Sim. ->Walk. Sim. ->Walk. Sim. ->Walk. Walk. ->Sim.
Attributes ObjectID,ID 3DRepresentative,Position ObjectID ObjectID,Startingtime,Destination-time, Destination-position ObjectID, TypeoInteraction f
Table 1: Examplesof some objects
4.1CouplingModule Thecouplingmodulecollectsmessages,whichstemf romtheconnectionstotheobjects ofthesimulation,andforwardsthosetothewalkth roughsystem.Furthermore,the objectIDhelpsdistributingthemessagesaboutuser interactions,whichcomeoutofthe walkthroughsystem,throughtheadequatechannelst othesimulator.Ourgoalforthe couplingmoduleistocapsulateallsimulationand simulationtoolsdependenttasks. Typicalimplementedtasks ofour system are: • Transfer of coordinates. • Storageoftheobjectcoordinates,e.g.duringmove mentofanobjectfrom sourcetodestination. • Storageothe f objectsizefor raightplacing itn hescene. • Initialisation,creation,anddeletionofmovingob jects.Thesimulationonly ceratethestartandendtimeothe f object. • Computation othe f speedomoving f objects. AllthecommunicationwillgothroughaTCP/IPconn ection(stream),whichhasbeen establishedonceinthebeginningothe f simulation Each . package whichhasbeensendis equivalentto produced a message(e.g.“createobje ct”o“move r object”).Itis 96bytebig andalwayscontainsthedata-fieldstypeom f essage o, bjectIDandstarting-timeotfhe objectinthesimulation.DependingontheobjectID data-fieldslikeexpiring-date, startingposition,endpositionorotation r etc.ma ybeadded.Forthereceiptathreadhas beenimplementedinthewalkthroughsystemwhichis runningparalleltothethreadfor therendering othe f walkthroughsystem.
4.2TimeSynchronisation Inconcretesimulationssimulationtimejumpsfrom other.Calculationshappenaone t pointofsimulati thesimulationtimeisrequiredtomoveforwardin simulationtimeprogressesintotalhavetobecalc withinthesimulatorshastobestoppedregularly. simulationtimeinordertopreventinconsistencies simulators.Tocompensatefortheuseocalculatio f forerun.Thereforeprogressesinthesimulationcan time.Alargeforerunallowsforthecompensationo load,butpreventsafastuserreactiontoevents. differentuserscenarios.Itwouldbeadvantageous
onepointofsimulationtimetothe ontime,thoughtheyneedrealtime.If (resp.proportionalto)realtime, ulatedfasterthanrealtime.Time Visualisingtimeshouldnotoutrun betweenvisualisingandthe ntime,simulatorshavetobe givena bevisualisedinrealtime=simulation bf iggervariationsin computation Varyingfastreactionsaresensiblein tokeepforeruntimeforthese
experimentsvariable.Ifvisualisingtimeapproache ssimulationtimecrucially,a deceleration rather than halt a of thevisualising helps theviewer toget caompletepicture ofthesystem.Visualisationsarecalculatingduet otherhythmoftheset-upofnew pictures.Thepicturerepetitionrateothe f visual isingshouldbde ecoupledfromthetime progress of simulation. standsforthesimulationtimeof Let t betherealtimeoof ursystem.Furthermore ts(t) oolsthere thesimulatorand tw(t)forthewalkthroughsystemtime.Withsimulationt isnotlinearto t, whichmeansthatit oftenoccurstheproblemthatsimulationtime ts(t) doesnapply ot toalltimes t:t=t s(t). Inmanysimulationtoolswalkthroughsystemtime runs in partly a constant,stair-likefunction (seefig. 4). ts(t),t w(t)
t ts(t)
event k event j event i
} computation timefor event
j t
Figure 4:
Realtime andsimulationtime
Thesimulatorwillalwaystrytokeepclosetoreal time t. Thisproblemwillintensifyina systemwithwalkthrough a systemwhichibsi-direct ionallycoupled,astheentiresystem tw(t). This doesnotonlyincludesimulationtime ts(t) butalsowalkthroughsystemtime may causeproblems becauseovarious f reasons: • Delaythroughtransmission: Awalkthroughsystemwhichisrunningon anothercomputercauseshightransmissiontimes.Th esemayresultindelayson both t computers,whensimulationstatusointerac r tionspecificationsothe f user aresentfrom onecomputer totheother. • Divergingotfimebases: Therelativetimebasisothe f hardwaretimerinbo th computersmaydivergeinthecourseotime f andthe reforecomplicateanexact synchronisation. Thismeansthatitwillgetveryhardtoguarantee for alltimes: t:t w(t)=t s(t).Theterm sbehindthe follow-uptime ∆tswstandsforthetimewhichthewalkthroughtimelag the time which the walkthrough simulationtime.Correspondingly, ∆trepresents w system lags behindtherealtime: = ts(t) t- w(t) ∆tw(t) = t t- w(t) ∆tsw(t) Follow-uptimes causeproblemsfortheuseothe f simulator,ifthe simulationprocedure ischangingduetoaspecificationoftheviewerin thewalkthroughsystem.Besides,it identified: may getproblematicithe f progression osimulatio f tnime ts(t) isnotclearly tw(t)
