Excerpt from imbWBIWeb Business Intelligence libraries of imbVeles Framework..ConsoleTool Console command line reference. Implementation class (CommandConsole) of (imbWBIWeb Business Intelligence libraries of imbVeles Framework..ConsoleTool, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null 1.0.0.0). Generated: Monday, November 19, 2018 – imbWBIWeb Business Intelligence libraries of imbVeles Framework. Console tool [v0.3.1]. Author: Goran Grubić [Copyright (c) 2017-2018]. Generated with imbACE.Core (0.2.0.21)
imbWBI Console Tool v0.3.1 Console Reference026 Plugin: itm
Class imbWBIWeb Business Intelligence libraries of imbVeles Framework..IndustryTermModel.consolePlugin.itmPlugin
imbWBIWeb Business Intelligence libraries of imbVeles Framework..ConsoleTool Console.itmPlugin
This is imbACE advanced console plugin for itmPlugin
1 itm.CloneFVEOnSTX
Ensures consistant settings accross FVERefers to Feature Vector Extraction model, that may have one or more Feature Vector Providers, which is implementation wrapper for particular document semantic similarity computation model.... models in the experiment
It will take first FVERefers to Feature Vector Extraction model, that may have one or more Feature Vector Providers, which is implementation wrapper for particular document semantic similarity computation model.... model, and make clones with different Semantic Expansion setting – other FVEs will be removed.
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | name | String | * | Name of the experiment to process, or * to use currently selected |
02 | start | Int32 | 3 | Starting value for StxSemantic Term Expansion, number of steps that a lemma (term) will be expanded using Semantic Cloud, part of Cosine SSRM computation model - semantic similarity computation, applied in Industry Term Model (imbWBI)... |
03 | end | Int32 | 8 | Ending value for StxSemantic Term Expansion, number of steps that a lemma (term) will be expanded using Semantic Cloud, part of Cosine SSRM computation model - semantic similarity computation, applied in Industry Term Model (imbWBI)... |
04 | code | String | 3 or 4 letter code indicating how the settings are different then in other experiments,* means sufix will be created automatically according to other modifications | |
05 | kFold | Int32 | -1 | number of k-folds, when specified it overrides settings of the experiment |
06 | rnd | Int32 | 0 | Overrides sample randomization, 0:no change, 1:randomize true, -1:randomize false |
07 | rename | String | Renames the experiment, if * the name is set automatically | |
08 | mode | String | stxSemantic Term Expansion, number of steps that a lemma (term) will be expanded using Semantic Cloud, part of Cosine SSRM computation model - semantic similarity computation, applied in Industry Term Model (imbWBI)... | Clone modes: stxSemantic Term Expansion, number of steps that a lemma (term) will be expanded using Semantic Cloud, part of Cosine SSRM computation model - semantic similarity computation, applied in Industry Term Model (imbWBI)..., lpfCloud Matrix mechanism: Low Pass Filter, that reduces Term Weight or removes the term completely from Semantic Cloud of a Category, based on Cloud Frequency (CF) - number of Categories having the same term in the Cloud.... |
Example :
itm.CloneFVEOnSTX name="*";start=3;end=8;code="";kFold=-1;rnd=0;rename="";mode="stx";
2 itm.Compose
Utilize composite template system to create an experiment instance
What it load component specified and set the experiment as current
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | shell | String | default | Name of experiment schell definition from composite templates sub folder of the project |
02 | classifiers | String | default | Name of classifier set XML file from composite template sub folder of the project |
03 | fveRefers to Feature Vector Extraction model, that may have one or more Feature Vector Providers, which is implementation wrapper for particular document semantic similarity computation model.... | String | default | Name of the Feature Vector Extractor XML file from composite template sub folder of the project |
Example :
itm.Compose shell="default";classifiers="default";fve="default";
3 itm.CrawlScript
Creates script to perform (re)build MCRepository crawl
Creates script that prepares crawler, crawl job and performs the crawl
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | name | String | inIndustry | What component of the model to crawl |
02 | clearRepo | Boolean | False | If true it will clear existing MC RepoValues: True,False |
03 | debug | Boolean | True | If true it will show some extra information during the processValues: True,False |
04 | autorun | Boolean | True | If true it will autorun created script without asking userValues: True,False |
Example :
itm.CrawlScript name="inIndustry";clearRepo=False;debug=True;autorun=True;
4 itm.Diagnostic
Evaluates current Industry Term Model project’s state
It will check in which phase the current project is, and if it is ready for application
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | verbose | Boolean | True | If true it will produce more in-detail report on the state of the current projectValues: True,False |
Example :
itm.Diagnostic verbose=True;
5 itm.DiagnosticMode
Sets configuration into system-check mode where minimum computation load is run, in single thread, just to enable debugging and system health
It will: set to single thread execution, one web page per web site is loaded, k is set to 1 (k-fold validation), uses cached lexic resource.
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | toggle | Int32 | 1 | if value 0, nothing changed, if its -1 it will turn off the diagnostic mode, if its 1 it will turn on the diagnostic mode |
Example :
itm.DiagnosticMode toggle=1;
6 itm.Experiment
Executes currently selected experiment
It will create experiment execution context and run
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | classReporting | Boolean | True | If true it will create additional report on each industry / categoryValues: True,False |
02 | caseReporting | Boolean | True | If true it will create additional report for each processed caseValues: True,False |
03 | resultReporting | Boolean | True | if true it will create additional report for each validation caseValues: True,False |
04 | makeGeneralReport | Boolean | True | if true it will create General reportsValues: True,False |
05 | sufix | String | Adds specified sufix to experiment report folder name |
Example :
itm.Experiment classReporting=True;caseReporting=True;resultReporting=True;makeGeneralReport=True;sufix="";
7 itm.ExperimentMatrix
Automatically creates 3×3 matrix of experiments variating TWRefers to HTML Tag induced term weight factor, applied during Lemma Table construction (Industry Term Model, imbWBI)..., TCRefers to Term Category induced term weight factor, in context of Industry Term Model (imbWBI)... and RXRefers to Continual Filter - Weight Reduction, in context of Semantic Cloud Constructor, Industry Term Model (imbWBI)... options
Creates experiment matrix using LPFCloud Matrix mechanism: Low Pass Filter, that reduces Term Weight or removes the term completely from Semantic Cloud of a Category, based on Cloud Frequency (CF) - number of Categories having the same term in the Cloud...., DFCDocument Frequency Factor - multiplier that is applied to Number of Documents variable in TF-cIDF term weight model. If DFC = 0, then it is TF model, if DFC=1, then it is typical TF-IDF...., IDFOn specified
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | LPFCloud Matrix mechanism: Low Pass Filter, that reduces Term Weight or removes the term completely from Semantic Cloud of a Category, based on Cloud Frequency (CF) - number of Categories having the same term in the Cloud.... | Int32 | 2 | Low pass frequecy– |
02 | DFCDocument Frequency Factor - multiplier that is applied to Number of Documents variable in TF-cIDF term weight model. If DFC = 0, then it is TF model, if DFC=1, then it is typical TF-IDF.... | Double | 5 | Document Frequency Correction |
03 | IDFOn | Boolean | True | IDF on / offValues: True,False |
04 | fveRefers to Feature Vector Extraction model, that may have one or more Feature Vector Providers, which is implementation wrapper for particular document semantic similarity computation model.... | String | CSSRM | FVERefers to Feature Vector Extraction model, that may have one or more Feature Vector Providers, which is implementation wrapper for particular document semantic similarity computation model.... name |
05 | shell | String | k1 | Shell name |
06 | stxStart | Int32 | 2 | StxSemantic Term Expansion, number of steps that a lemma (term) will be expanded using Semantic Cloud, part of Cosine SSRM computation model - semantic similarity computation, applied in Industry Term Model (imbWBI)... start |
07 | stxEnd | Int32 | 6 | StxSemantic Term Expansion, number of steps that a lemma (term) will be expanded using Semantic Cloud, part of Cosine SSRM computation model - semantic similarity computation, applied in Industry Term Model (imbWBI)... end |
08 | StrictPOS | Boolean | False | Strict POSPart-of-speech, is very frequently used to provide linguistic information to NER and CR in form of features in statistical approaches... PolicyValues: True,False |
09 | Report | Boolean | False | Should generate general report on classesValues: True,False |
Example :
itm.ExperimentMatrix LPF=2;DFC=5;IDFOn=True;fve="CSSRM";shell="k1";stxStart=2;stxEnd=6;StrictPOS=False;Report=False;
8 itm.ExperimentRange
What is purpose of this?
What it will do?
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | TWRefers to HTML Tag induced term weight factor, applied during Lemma Table construction (Industry Term Model, imbWBI)... | String | std | Term Weight Flag for HTML Tags |
02 | TCRefers to Term Category induced term weight factor, in context of Industry Term Model (imbWBI)... | String | std | Term Category |
03 | RXRefers to Continual Filter - Weight Reduction, in context of Semantic Cloud Constructor, Industry Term Model (imbWBI)... | String | div | Redux option |
04 | DFCDocument Frequency Factor - multiplier that is applied to Number of Documents variable in TF-cIDF term weight model. If DFC = 0, then it is TF model, if DFC=1, then it is typical TF-IDF.... | String | 1.5,2.0,3.5,5.0,7.5,10 | DFCDocument Frequency Factor - multiplier that is applied to Number of Documents variable in TF-cIDF term weight model. If DFC = 0, then it is TF model, if DFC=1, then it is typical TF-IDF.... values |
05 | LPFCloud Matrix mechanism: Low Pass Filter, that reduces Term Weight or removes the term completely from Semantic Cloud of a Category, based on Cloud Frequency (CF) - number of Categories having the same term in the Cloud.... | String | 2 | LPFCloud Matrix mechanism: Low Pass Filter, that reduces Term Weight or removes the term completely from Semantic Cloud of a Category, based on Cloud Frequency (CF) - number of Categories having the same term in the Cloud.... values |
06 | IDFOn | Boolean | True | IDF turn on and offValues: True,False |
07 | fveRefers to Feature Vector Extraction model, that may have one or more Feature Vector Providers, which is implementation wrapper for particular document semantic similarity computation model.... | String | CSSRM | FVERefers to Feature Vector Extraction model, that may have one or more Feature Vector Providers, which is implementation wrapper for particular document semantic similarity computation model.... name |
08 | shell | String | k1 | Shell name |
09 | stxStart | Int32 | 2 | StxSemantic Term Expansion, number of steps that a lemma (term) will be expanded using Semantic Cloud, part of Cosine SSRM computation model - semantic similarity computation, applied in Industry Term Model (imbWBI)... start |
10 | stxEnd | Int32 | 6 | StxSemantic Term Expansion, number of steps that a lemma (term) will be expanded using Semantic Cloud, part of Cosine SSRM computation model - semantic similarity computation, applied in Industry Term Model (imbWBI)... end |
11 | StrictPOS | Boolean | False | Strict POSPart-of-speech, is very frequently used to provide linguistic information to NER and CR in form of features in statistical approaches... PolicyValues: True,False |
12 | Report | Boolean | False | Should generate general report on classesValues: True,False |
13 | rnd | Int32 | 0 | Overrides sample randomization, 0:no change, 1:randomize true, -1:randomize false, more than 1 will repeat the same model specified number of times |
14 | exName | String | * | Experiment name |
15 | classifier | String | default | Name of classifier compound |
Example :
itm.ExperimentRange TW="std";TC="std";RX="div";DFC="1.5,2.0,3.5,5.0,7.5,10";LPF="2";IDFOn=True;fve="CSSRM";shell="k1";stxStart=2;stxEnd=6;StrictPOS=False;Report=False;rnd=0;exName="*";classifier="default";
9 itm.GetExperiment
Provides new experiment or loads existing
It will load or create new experiment and set is as the active one, ready for execution
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | name | String | Exp01 | name of the experiment |
02 | makeDefault | Boolean | True | if true it will create experiment setup with default settings (if experiment not found)Values: True,False |
03 | debug | Boolean | True | if true it will provide more verbose outputValues: True,False |
Example :
itm.GetExperiment name="Exp01";makeDefault=True;debug=True;
10 itm.ImportExperiment
Allows reproduction of earlier experiment by loading experimentSetup.xml from report folder.
It will load the experiment from specified address and set it as current experiment in the console, optionally it can run the experiment immediatly.
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | path | String | reports\STAGE05 | Path pointing to the folder with experiment report. You don’t need to specify exact file name, just directory path. The path is relative to application installation folder |
02 | execute | Boolean | True | If true it will automatically execute loaded experimentValues: True,False |
03 | silent | Boolean | False | If true, it will apply default answers if otherwise user input was required. This is intended for scripted/automatic executionValues: True,False |
04 | suffix | String | _redone | Report suffix used for report folder name (it will also filter out targeted experiments, in case you are pointing at folder – to avoid endless execution loop) |
05 | onlyWithError | Boolean | False | Execute only if experiment had errorsValues: True,False |
Example :
itm.ImportExperiment path="reports\STAGE05";execute=True;silent=False;suffix="_redone";onlyWithError=False;
11 itm.Modify
Adjusting experiment settings
Modifies current experiment
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | TWRefers to HTML Tag induced term weight factor, applied during Lemma Table construction (Industry Term Model, imbWBI)... | String | std | Term Weight Flag for HTML Tags |
02 | TCRefers to Term Category induced term weight factor, in context of Industry Term Model (imbWBI)... | String | std | Term Category |
03 | RXRefers to Continual Filter - Weight Reduction, in context of Semantic Cloud Constructor, Industry Term Model (imbWBI)... | String | div | Redux option |
04 | LPFCloud Matrix mechanism: Low Pass Filter, that reduces Term Weight or removes the term completely from Semantic Cloud of a Category, based on Cloud Frequency (CF) - number of Categories having the same term in the Cloud.... | Int32 | 2 | Term Weight Flag for HTML Tags |
05 | DFCDocument Frequency Factor - multiplier that is applied to Number of Documents variable in TF-cIDF term weight model. If DFC = 0, then it is TF model, if DFC=1, then it is typical TF-IDF.... | Double | 1.1 | DF Correction |
06 | IDFOn | Boolean | True | IDF turn on and offValues: True,False |
07 | StrictPOS | Boolean | False | Strict POSPart-of-speech, is very frequently used to provide linguistic information to NER and CR in form of features in statistical approaches... PolicyValues: True,False |
Example :
itm.Modify TW="std";TC="std";RX="div";LPF=2;DFC=1.1;IDFOn=True;StrictPOS=False;
12 itm.Open
Sets the current Industry Term Model projects
It will try to load the project or to create new under specified name
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | name | String | itm01 | Name of project to load |
Example :
itm.Open name="itm01";
13 itm.RepoOptimization
Removes unused content from repositories
It will use current project settings to filter repo entries for pages on language of preference and to delete irrelevant content
Example :
itm.RepoOptimization
14 itm.Save
Saves the currently selected Industry Term Model project
It will call for save of the currently selected Industry Term Model project
Example :
itm.Save
15 itm.Setup
Performs basic setup procedure, should be called after new project created
It will ask user to insert or confirm each property in configuration modules: crawlSetup, nlpRepoProcessSetup, etc…
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | verbose | Boolean | True | Provides additional hints and explanations during the setup processValues: True,False |
Example :
itm.Setup verbose=True;
16 itm.SingleFoldTest
Trains the classifier and performs classification over complete industry classes, used for debug purposes
It will use complete sample of the class to train classifier, later it will run classification on the same class
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | prefix | String | Test name prefix | |
02 | model | itmModelEnum | chunkTFIDF | Model to use for training and classificationValues: none,wordTFIDF,chunkTFIDF,wordTFIDF_complex,semantic |
03 | report | Boolean | False | if true it will create extensive debug informationValues: True,False |
04 | debug | Boolean | True | if true it will create extensive debug informationValues: True,False |
Example :
itm.SingleFoldTest prefix="";model=chunkTFIDF;report=False;debug=True;
17 itm.Summary
Scans report folders to produce summary report on all experiments
Scans for experimentSetup.xml files and collects all FVERefers to Feature Vector Extraction model, that may have one or more Feature Vector Providers, which is implementation wrapper for particular document semantic similarity computation model.... data into single table
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | reportName | String | Summary | Prefix to insert in name |
02 | subfolder | String | Subfolder within report directory |
Example :
itm.Summary reportName="Summary";subfolder="";
18 itm.Test
Used for development, to run quick test
on what ever
Command arguments:
ID | Name | Type | Default | Comment |
---|---|---|---|---|
01 | word | String | word | — |
02 | steps | Int32 | 5 | — |
03 | debug | Boolean | True | –Values: True,False |
Example :
itm.Test word="word";steps=5;debug=True;
General info
Type ‘help’ and press enter to show list of all commands, ‘exit’ to close the console.
Command with default parameters: [command name]
For help on a command: [command name] ?
To be prompted for parameters: [command name] *