How to confirm conditions of analysis and check execution logs
Table of contents
Confirm execution option after a pipeline run
- You can confirm option values in the three views, 'Module Flow View', 'Analysis Status - Progress' and 'Analysis Status - Result'.
- How to open these views are described here.
- A series of processes to check option values is identical in three views.
A simplest example
- In this example, how to check option values in 'Module Flow View' is described.
- After open a 'Module Flow View', click a module icon you are interested in.
- A module detail sub-window is displayed.
- In this project, there are two analysis histories of 'FastQC (single-end)' pipelines with different option and an analysis history of 'BWA (SE) + GE'.
- You can see the option value difference in the two different options in FastQC analysis module history.
Option condition With '--nogroup' option With default option 'Analysis - Option' views
When the pipelines start running
*
snip
*
*
snip
*
Module detail sub-windows - You can see a difference of option value under a borderline in these module detail sub-windows.
- Note that FastQC version values are FastQC-0.10.1 in both executed modules.
- In the right sub-window, an 'End Date' value, 'Output Dataset' value and 'LOG' icon at the bottom, are absent because this analysis is not finished.
- This 'FastQC (SE)' pipeline constitute of one module and one tool.
More complex example
- A pipeline 'BWA (SE) + GE' constitute of 2 modules and total 5 tools.
*
snip
*
- This hierarchy is also recognized in the option description of this pipeline.
- You can see option values per a tool in the two module sub-windows.
- Note that an option 'GENOME_ID' and an option 'samtools version' appear many times in the sub-windows but one time each option in the 'Analysis - Option' view. Same option can be aggregated in a pipeline and these aggregated options are listed at the top of the 'Analysis Option' view.
- 'Number of thread' option in the first tool 'bwa aln (single-end)' of the first module 'BWA (SE)' do not appear on the 'Analysis – Option' view. This is a hidden option which cannot set by the user.
Check the execution logs
- You can see execution logs from the three views, 'Module Flow View', 'Analysis Status - Progress' and 'Analysis Status - Result'.
- How to open these views are described here.
- A series of processes to check execution logs is identical in three views.
A simplest example
- In this example, how to check option values in 'Module Flow View' is described.
- After open a 'Module Flow View', click a module icon you are interested in and click a log icon from the displayed module detail sub-window.
-
- 'Runtime Log' window is displayed.
- The following table is the explanation for each file.
# Name example Description 1 DP00047852.demouser01.e300902.1 A standard error log 2 DP00047852.demouser01.e300902.1.txt An alias of the above file, which can open with many browsers at direct
Remember that error causes is frequently described in this file when a module ends with an 'Abort' status.3 DP00047852.demouser01.o300902.1 A standard output log 4 DP00047852.demouser01.o300902.1.txt An alias of the above file, which can open with many browsers at direct
Remember that error causes is frequently described in this file when a module ends with an 'Abort' status.5 DP00047852.demouser01.pe300902.1 No meaning for users. 6 DP00047852.demouser01.pe300902.1.txt No meaning for users. 7 DP00047852.demouser01.po300902.1 No meaning for users. 8 DP00047852.demouser01.po300902.1.txt No meaning for users. 9 evaluated.txt A log file about executed commands in one line format. 10 evaluated_command.txt A log file about executed commands in a human readable format. 11 input_check A log file about the number of input files. 12 myscript A local script to use only in this tool execution. 13 time.log.txt A log file about running time.
- A file example of DP00047852.demouser01.o300902.1.txt
- Contents of this file is totally depend on the executed tools.
- Sometimes this file is empty.
- A file example of DP00047852.demouser01.e300902.1.txt
- Contents of this file is totally depend on the executed tools.
- In many cases this file is empty when the tools correctly ended.
- A file example of evaluated_command.txt
- A file example of time.log.txt
- In this file, real execution start time and execution end time are described.
- Between the start time and the end time it does not contain system operation time and the waiting time for starting each tool.
More complex example
- A pipeline 'BWA (SE) + GE' constitute of 2 modules and total 5 tools.
- After open a 'Module Flow View', click a module about this pipeline and click a log icon from the displayed module detail sub-window.
- 'Runtime Log' view about 'BWA (SE)' (the former module) which constitute of 3 tools.
- Note that there are 3 sets of log files in 3 directories '1_C000000001', '2_C000000002' and '3_C000000003'.
- Each of directories corresponds with the 3 tools 'bwa aln (single-end)', 'bwa samse (single-end)' and 'samToBam/sort/indexing SAMtools with head'.
- Meaning of files in each log directory is same as described here.
- 'Runtime Log' view about 'loadBAM2ge_db (single-end)' (the latter module) which constitute of 2 modules.
- Note that there are 2 sets of log files in 2 directories '1_C000000001' and '2_C000000002'.
- Each of directories corresponds with the 2 tools 'bam2ge_tsv (single-end)' and 'load_gbmap_data.pl (directional/non-directional)'.
- Meaning of files in each log directory is same as described here.