STEPlib SysML: How To use the IS Core Model Scripts¶
This page explains how to use the Scripts of the STEP architecture to generate the documentation for the IS for Core Model. Other pages (in progress) explain the process for other APs which include content for more of the architecture.
The image below shows the models in the STEP Extended Architecture that are used by, and generated for, the International Standard documentation.
Figure 129 STEP Extended Architecture showing usage in this set of scripts
Contents:
Prerequists¶
Warning
work in progress
DO A SPELL CHECKING BEFORE EXPORTING.
Needs:
java (saxon and tagsoup jar provided)
Python json.tool (for pretty print of JSON schema)
Ruby (for generation of Canonical XMI) and additional Gems:
Ruby 2.5.3 (using an internet connection without proxy/firewall):
gem install mini_portile2 gem install systemu gem install macaddr gem install uuid gem install rubyzip gem install nokogiri
Commands are assuming executingting in cmd window on PC.
The add to path if not already there.
Figure 130 Add ruby to path environment variable
Process explained¶
Attention
The Core model reuses the scripts for generating the 4xxx documentation using the core model in place of the domain model. It does not use the scripts for 2xx documents, nor the scripts for implementation models.
The image below shows the models used and generated by the scripts. It includes a step-by-step view of the scripts where the yellow numbers refer generation of files that are not HTML, and the purple numbers are the HTML files.
Figure 131 IS Document Generation Process
Start. The XHTML files generated externally to this process are copied into the output folders
- The diagram images are exported from the SysML tool as png files. In MagicDraw this is done with a macro STEPlib/utils/MagicDraw/exportDiagramsMacro.js;
- The tool SysML files are converted into Canonical XMI and DI using a tool dependent script;
- The settings for the scripts are contained in ExtractID/VariablesData243.xml and AntPub/VariablesData243.xml. These should be updated or new variants created for other APs or core model. These are input arguments to the scripts.;
- ExtractID/CanonicalGenerateIDsXML.xsl: this script generates temporary file (sysml_models_ids.xml) containing basic information on ids and names for all entities in the models. This is used to resolve any external references instead of having to open every the Canonical XMI and DI for all the scripts;
- ExtractID/CanonicalExtract_ID.xsl: For each of the models this generates the image html files including the image maps. These are then copied into the appropriate Antpub/output/data folder;
- AntImp/CanonicalDomainModel_xmi_to_xsd.xsl: This generates the XSD for the model (note this imports AntPub/FunctionsCanonical.xsl and AntImp/FunctionsCanonical.xsl). The output is then sorted using AntImp/DomainModel_sort_xsd.xsl ;
- AntImp/DomainModel_xmi_to_json.xsl: This generates the XSD for the model which is then formatted using python -m json.tool (note the xsl imports AntPub/FunctionsCanonical.xsl);
The HTML documents that are generates with the following steps (see purple numbers in figure above):
The clauses that are not dependent of any other HTML document are generated first as XHTML working files:
- AP Clause 4 CanonicalAPClause4_Documentations.xsl
- AP Clause 6 (for Uof, CO and CC model) CanonicalConformanceModel_Documentations.xsl or for single table used CanonicalConformanceModelSingle_Documentations.xsl
- AP AAM annex CanonicalAAM_Documentations.xsl
- DM/Core Clause 4 CanonicalDomainModel_clause4.xsl
- DM/Core Clause 5 CanonicalDomainModel_clause5.xsl
- Clause 3 TermsDefs.xsl
- Clause 2 and Biblography NormRefs_Biblio.xsl
- DM/Core Annex BBD list DomainModel_annex_bom_expg
- DM/Core Clause 4.3 RD ReferenceData.xsl
- DM/Core model index CanonicalDomainModel_index.xsl
All the XHTML files (including those from STEPmod) are read to generate the index that will appear at the end of the table of contents. The script locates all heading tags in each file with class=(index_entry index_entry_type_{xxx}) where {xxx} is used to filter the index. It assumes a format of “{clause.number} {clause title}”, the {clause.number} us used as the text for the hyperlink and the {clause title} is prefixed before adding to the index.
- ToC index Canonical_index.xsl
All the XHTML files (including those from STEPmod) are rebuilt and output as working XML with corrected figure numbers; table numbers; note and example numbers. The script locates all tags in each file with class = figure or table that are not inside a tabcontent, and prefixes the figure or table number to the tag with class = figureTitle or tableTitle. . It finds all tags with class = note or example and if more than one between heding tags applies numbering.
- renumber_figurestables.xsl
The working XML files are read to generate the home.htm with the ToC, ToF and ToT for both AP and DM (or Core). The script locates all headings to add to the table of contents, all tags with class=”figureTitle” are added to the table of figures and all tags with class=”tableTitle” are added to the table of tables.
- CanonicalDomainModel_Home.xsl
All the working XML files are then converted into HTML files
- xhtml_to_htm.xsl
The XHTML and XML working files are deleted.
The Script folders are:
ExtractID
Functions.xsl (common functions)
CanonicalGenerateIDsXML.xsl
- input xml: ExtractID/VariablesDataXXX.xml this has all the xmi and di filenames
- imports ExtractID/Functions.xsl
- generates ExtractID/output/sysml_models_ids.xml. This is copied to AntPub/input/
CanonicalExtract_ID/xsl
- input xml: ExtractID/VariablesDataXXX.xml
- input param: model_to_generate (AAM, CM, DM, CC, SM, Core)
- uses ExtractID/output/sysml_models_ids.xml
- generates ExtractID/output/html/*.htm which should then be moved to AntPub/output/data/{subfolder(s)}/
AntImp
FunctionsCanonical.xsl
DomainModel_canonicalxmi_to_xsd.xsl
- input xml: DomainModel.xmi (Canonicial XMI)
- input param: variablesFileName=../AntPub/VariablesDataXXX.xml
- reads AntPub/input/sysml_models_ids.xml
- imports AntPub/FunctionsCanonical.xsl and AntImp/FunctionsCanonical.xsl
- generates (as > argument) unsorted_DomainModel_from_cxmi.xsd
DomainModel_sort_xsd.xsl
- input xml: unsorted_DomainModel_from_cxmi.xsd
- generates (as > argument) DomainModel.xsd which should then be moved to AntPub/output/data/domain_models/{APShortName}/
DomainModel_canonicalxmi_to_json.xsl (imports AntPub/FunctionsCanonical.xsl)
- input xml: DomainModel.xmi (Canonicial XMI)
- input param: variablesFileName=../AntPub/VariablesDataXXX.xml
- input param: combined (0 or 1) - default 1.
- input param: fileNamePost. default = 243
- input param: debuggingMode (0 1)
- reads AntPub/input/sysml_models_ids.xml
- imports AntPub/FunctionsCanonical.xsl
- generates (as > argument) DomainModel_NotPretty{fileNamePost}.json
AntPub
FunctionsCanonical.xsl
- input param: variablesFileName (default AntPut/VariablesData.xml this must always be changed)
- reads AntPub/input/sysml_models_ids.xml
CanonicalFunctionsHTML.xsl
- input param: model_to_generate (AP, DM, Core)
- input param: my_model (AAM, CM ,CC, SM, Core) default is model_to_generate
- includes AntPub/FunctionsCanonical.xsl
CanonicalAPClause4_Documentations.xsl
- input xml Data_planning_model.xmi
- input param: model_to_generate=AP
- input param: variablesFileName
- reads AntPub/input/sysml_models_ids.xml
- includes AntPub/CanonicalFunctionsHTML.xsl
- generates AntPub/output/data/application_protocols/{APShortName}/sys/4_info_reqs.xhtml
CanonicalConformanceModel_Documentations.xsl
- input xml Domain_model_conformance.xmi
- input param: model_to_generate=AP
- input param: variablesFileName
- reads AntPub/input/sysml_models_ids.xml
- includes AntPub/CanonicalFunctionsHTML.xsl
- generates AntPub/output/data/application_protocols/{APShortName}/sys/6_ccs.xhtml
CanonicalConformanceModelSingle_Documentations.xsl
- input xml Domain_model.xmi
- input param: model_to_generate=AP
- input param: variablesFileName
- reads AntPub/input/sysml_models_ids.xml
- includes AntPub/CanonicalFunctionsHTML.xsl
- generates AntPub/output/data/application_protocols/{APShortName}/sys/6_ccs.xhtml
CanonicalAAM_Documentations.xsl
- input xml Activity_model.xmi
- input param: model_to_generate=AP
- input param: variablesFileName
- reads AntPub/input/sysml_models_ids.xml
- imports AntPub/CanonicalFunctionsHTML.xsl
- generates AntPub/output/data/application_protocols/{APShortName}/sys/annex_aam.xhtml
CanonicalDomainModel_clause4.xsl
- input xml Domain_model.xmi (or Core_model.xmi)
- input param: model_to_generate=DM (or Core)
- input param: variablesFileName
- input param: split4 (default 0 which will ouput all clause 4 in one file. change to 1 for clause 4 split by packages)
- input param: splitModel (default 0 indicating the all is contained in the input xmi. Set to 1 if multiple xmi (e.g. for core)
- reads AntPub/input/sysml_models_ids.xml
- includes AntPub/CanonicalFunctionsHTML.xsl
- generates AntPub/output/data/domain_models/{APShortName}/sys/4_info_reqs_XXX.xhtml
CanonicalDomainModel_clause5.xsl
- input xml Domain_model.xmi (or Core_model.xmi)
- input param: model_to_generate=DM (or Core)
- input param: variablesFileName
- input param: splitModel (default 0 indicating the all is contained in the input xmi. Set to 1 if multiple xmi (e.g. for core)
- reads AntPub/input/sysml_models_ids.xml
- includes AntPub/CanonicalFunctionsHTML.xsl
- generates AntPub/output/data/domain_models/{APShortName}/sys/5_main.xhtml
TermsDefs.xsl
- input xml AntPub/VariablesDataXXX.xml
- input param: model_to_generate (AP, DM or Core)
- input param: termtag (APterms DMterms) - tag containing the terms to use in the input xml
- input param: variablesFileName
- includes AntPub/CanonicalFunctionsHTML.xsl
- generates AntPub/output/data/{Core, AP or DM}/sys/3_defs.xhtml
NormRefs_Biblio.xsl
- input xml AntPub/VariablesDataXXX.xml
- input param: model_to_generate (AP, DM or Core)
- input param: nr_or_biblio (nr biblio) - tag containing the terms to use in the input xml
- input param: variablesFileName
- includes AntPub/CanonicalFunctionsHTML.xsl
- generates AntPub/output/data/{Core, AP or DM}/sys/2_refs.xhtml (filename specified in input xml)
- generates AntPub/output/data/{Core, AP or DM}/sys/biblio.xhtml (filename specified in input xml)
DomainModel_annex_bom_expg.xsl
- input xml AntPub/input/sysml_models_ids.xml
- input param: model_to_generate (DM or Core)
- input param: termtag=DMterms
- input param: variablesFileName
- includes AntPub/CanonicalFunctionsHTML.xsl
- generates AntPub/output/data/{Core or DM}/sys/annex_bom_expg.xhtml
ReferenceData.xsl
- input xml AntPub/VariablesDataXXX.xml
- input param: model_to_generate (DM or Core)
- input param: variablesFileName
- reads all reference data files as specified in input xml
- includes AntPub/CanonicalFunctionsHTML.xsl
- generates AntPub/output/data/{Core or DM}/sys/4_info_reqs_refdata.xhtml
CanonicalDomainModel_index.xsl
- input xml AntPub/VariablesDataXXX.xml
- input param: model_to_generate (DM or Core)
- input param: variablesFileName
- reads AntPub/input/sysml_models_ids.xml
- includes AntPub/CanonicalFunctionsHTML.xsl
- generates AntPub/output/data/{Core or DM}/sys/index_domain_model.xhtml
Canonical_index.xsl
- input xml AntPub/VariablesDataXXX.xml
- input param: model_to_generate (AP, DM or Core)
- input param: variablesFileName
- reads all the previously generated xhtml
- includes AntPub/CanonicalFunctionsHTML.xsl
- generates AntPub/output/data/application_protocols/{APShortName}/index_apdoc.xhtml
- generates AntPub/output/data/domain_models/{APShortName}/index_bomdoc.xhtml
- generates AntPub/output/data/core_model/index_bomdoc.xhtml
renumber_figurestables.xsl
CanonicalDomainModel_Home.xsl
- input xml Domain_model.xmi or Core_model.xmi
- input param: model_to_generate (DM or Core)
- input param: variablesFileName
- reads all output *.xml (from renumber_figurestables.xsl)
- includes AntPub/CanonicalFunctionsHTML.xsl
- generates AntPub/output/data/application_protocols/{APShortName}/home.htm (if model_to_generate=DM)
- generates AntPub/output/data/domain_models/{APShortName}/home.htm (if model_to_generate=DM)
- generates AntPub/output/data/core_model/home.htm (if model_to_generate=Core)
xhtml_to_htm.xsl
Export images from MagicDraw¶
Important
The image output settings must be set to 100%
File/Save as image/Image format/Options
Figure 132 set Export diagram size to 100%
Assuming a MagicDraw license is available:
In MagicDraw use the STEPlibutilsMagicDrawexportDiagramsMacro.js to generate the diagram images. see Load Macros for how to load and use macros. Open the Core_model, select the Core_model package and run the macro. This should create the png in the STEPlib/Core_model folder.
Figure 133 Export diagrams to png with macro
If there is no MagicDraw license, then the macros are not available. In this case the process is to export the diagrams using the inbuilt exporter, then generate and run a batch file which will rename the resultant images.
Open the Core_model then select File > Save As Image
Select all the diagrams and save as png to the model folder without dvlp, overwriting existing files:
Figure 134 Export diagrams to png using out-of-the-box MagicDraw
when executing the scripts, after the CanonicalGenerateIDsXML labeled rename images if not using the macro.
Todo
find out how to use macros with “reader” MagicDraw. Have checked with normal MagicDraw in evaluation model (i.e. no SysML license) and works OK.
Convert the SysML models to Canonical XMI¶
Todo
It starts from the mdzip. The first script generates unordered XMI, the second orders to create the Canonical XMI.
Drag and drop the Core_model folder onto the STEPlib/utils/MagicDraw/genXMIFromFolder.bat
- Drag and drop the Core_model/dvlp/core_model.mdzip onto the STEPlib/utils/MagicDraw/genXMIFromFolder.bat
- THIS DOES NOT WORK FOR ME - SO USE THE VERSION IN THE GIT
Or execute the following calls:
cd STEPlib/utils/MagicDraw
for /D %i in (../../Core_model/*) do (ruby "to_cxmi.rb" mdzip="../../Core_model/%i/dvlp/%~ni.mdzip" ruby "..\ExpToXMI\to_cxmi.rb" xmi="../../Core_model/%i/%~ni.xmi" ruby "..\ExpToXMI\to_cxmi.rb" xmi="../../Core_model/%i/%~ni.di")
//ruby "to_cxmi.rb" mdzip="../../Core_model/ProductDataManagement/dvlp/ProductDataManagement.mdzip"
//ruby "..\ExpToXMI\to_cxmi.rb" xmi="../../Core_model/ProductDataManagement/ProductDataManagement.xmi"
//ruby "..\ExpToXMI\to_cxmi.rb" xmi="../../Core_model/ProductDataManagement/ProductDataManagement.di"
Generate All - batch¶
Todo
convert this into batch script with arguments for {core_model} and {Core} and hardcoded relative paths {Harmo}
Execute the following calls (replacing your relative path to HarmoSTEPlib)::
del /q AntPub\output\data\core_model\* /s
rmdir /s /q AntPub\output\data\core_model
mkdir AntPub\output\data\core_model
mkdir AntPub\output\data\core_model\pictures
mkdir AntPub\output\data\core_model\refdata
mkdir AntPub\output\data\core_model\sys
mkdir AntPub\output\data\core_model\sys\script
xcopy AntPub\input\script AntPub\output\data\core_model\sys\script /s /e
powershell Compress-Archive -Path ..\Core_model\refdata\*.owl -DestinationPath AntPub\output\data\core_model\Core_model_RD.zip
xcopy ..\Core_model AntPub\output\Core_model\ /s /exclude:savezip.txt
powershell Compress-Archive -Path AntPub\output\Core_model -DestinationPath AntPub\output\data\core_model\Core_model.zip
del /q AntPub\output\Core_model /s
rmdir /s /q AntPub\output\Core_model
del /q Antpub\input\maps\*
cd ExtractID
del /q output\html\*
java -jar ../saxon9-8-0-6.jar -t VariablesDataCore.xml CanonicalGenerateIDsXML.xsl
java -jar ../saxon9-8-0-6.jar -t model_to_generate=Core VariablesDataCore.xml CanonicalExtract_ID.xsl
// rename images if not using macro
java -jar ../saxon9-8-0-6.jar -t model_to_generate=Core relativePath=..\..\Core_model\ ./output/sysml_models_ids.xml CanonicalRenameMDimages.xsl
move output\renameimages.bat ..\..\Core_model\.
..\..\Core_model\renameimages.bat
// copy and move images and sysml_models_ids
xcopy ..\..\Core_model\_*.png ..\AntPub\output\data\core_model\pictures
xcopy output\html\* ..\AntPub\input\maps
move output\html\* ..\AntPub\output\data\core_model
copy output\sysml_models_ids.xml ..\AntPub\input\. /y
cd ../AntPub
xcopy .\input\CoreSTEPmod_files\data\* .\output\data /s /e
//xcopy ..\..\utils\AntPub\input\CoreSTEPmod_files\data\* .\output\data /s /e
java -jar ../saxon9-8-0-6.jar -t model_to_generate=Core split4=1 splitModel=1 variablesFileName=VariablesDataCore.xml ../../Core_model/Core_model.xmi CanonicalDomainModel_clause4.xsl --suppressXsltNamespaceCheck:on 2>4000_4_error.log
java -jar ../saxon9-8-0-6.jar -t model_to_generate=Core variablesFileName=VariablesDataCore.xml splitModel=1 ../../Core_model/Core_model.xmi CanonicalDomainModel_clause5.xsl 2>4000_5_error.log
java -jar ../saxon9-8-0-6.jar -t model_to_generate=Core variablesFileName=VariablesDataCore.xml ./input/sysml_models_ids.xml termtag=DMterms DomainModel_annex_bom_expg.xsl
java -jar ../saxon9-8-0-6.jar -t model_to_generate=Core variablesFileName=VariablesDataCore.xml termtag=DMterms ./VariablesDataCore.xml TermsDefs.xsl
java -jar ../saxon9-8-0-6.jar -t model_to_generate=Core variablesFileName=VariablesDataCore.xml nr_or_biblio=nr ./VariablesDataCore.xml NormRefs_Biblio.xsl
java -jar ../saxon9-8-0-6.jar -t model_to_generate=Core variablesFileName=VariablesDataCore.xml nr_or_biblio=biblio ./VariablesDataCore.xml NormRefs_Biblio.xsl
java -jar ../saxon9-8-0-6.jar -t model_to_generate=Core variablesFileName=VariablesDataCore.xml ./VariablesDataCore.xml CanonicalDomainModel_index.xsl
java -jar ../saxon9-8-0-6.jar -t model_to_generate=Core variablesFileName=VariablesDataCore.xml ./VariablesDataCore.xml ReferenceData.xsl --suppressXsltNamespaceCheck:on
java -jar ../saxon9-8-0-6.jar -t model_to_generate=Core variablesFileName=VariablesDataCore.xml ./VariablesDataCore.xml Canonical_index.xsl
java -jar ../saxon9-8-0-6.jar -t model_to_generate=Core ./VariablesDataCore.xml renumber_figurestables.xsl --suppressXsltNamespaceCheck:on
java -jar ../saxon9-8-0-6.jar -t model_to_generate=Core variablesFileName=VariablesDataCore.xml ../../Core_model/Core_model.xmi CanonicalDomainModel_Home.xsl
java -jar ../saxon9-8-0-6.jar -t model_to_generate=Core ./VariablesDataCore.xml xhtml_to_htm.xsl
del /q output\data\core_model\sys\*.xhtml
del /q output\data\core_model\sys\*.xml
move from the AntPub/output/data/* to the <root folder>/iso10303_2xx/data:
move AntPub\output\data\* <root folder>\iso10303_243\data
Section author: Judith Crockford (AP243)