Platform for Drug Discovery


How to confirm conditions of analysis and check execution logs


Table of contents


Confirm execution option after a pipeline run


    • You can confirm option values in the three views, 'Module Flow View', 'Analysis Status - Progress' and 'Analysis Status - Result'.
    • How to open these views are described here.
    • A series of processes to check option values is identical in three views.

A simplest example

  • In this example, how to check option values in 'Module Flow View' is described.
  1. After open a 'Module Flow View', click a module icon you are interested in.
    • Image
    • A module detail sub-window is displayed.
    • In this project, there are two analysis histories of 'FastQC (single-end)' pipelines with different option and an analysis history of 'BWA (SE) + GE'.
  2. You can see the option value difference in the two different options in FastQC analysis module history.
    • Option conditionWith '--nogroup' optionWith default option
      'Analysis - Option' views
      When the pipelines start running
      Image
      *
      snip
      *
      Image
      Image
      *
      snip
      *
      Image
      Module detail sub-windowsImage Image
    • You can see a difference of option value under a borderline in these module detail sub-windows.
    • Note that FastQC version values are FastQC-0.10.1 in both executed modules.
    • In the right sub-window, an 'End Date' value, 'Output Dataset' value and 'LOG' icon at the bottom, are absent because this analysis is not finished.
  • This 'FastQC (SE)' pipeline constitute of one module and one tool.
    • Image

More complex example

    • Image
    • A pipeline 'BWA (SE) + GE' constitute of 2 modules and total 5 tools.
    • Image
      *
      snip
      *
      Image
    • This hierarchy is also recognized in the option description of this pipeline.
    • Image
    • You can see option values per a tool in the two module sub-windows.
    • Image
    • Note that an option 'GENOME_ID' and an option 'samtools version' appear many times in the sub-windows but one time each option in the 'Analysis - Option' view. Same option can be aggregated in a pipeline and these aggregated options are listed at the top of the 'Analysis Option' view.
    • 'Number of thread' option in the first tool 'bwa aln (single-end)' of the first module 'BWA (SE)' do not appear on the 'Analysis – Option' view. This is a hidden option which cannot set by the user.

Check the execution logs


    • You can see execution logs from the three views, 'Module Flow View', 'Analysis Status - Progress' and 'Analysis Status - Result'.
    • How to open these views are described here.
    • A series of processes to check execution logs is identical in three views.

A simplest example


  • In this example, how to check option values in 'Module Flow View' is described.
  1. After open a 'Module Flow View', click a module icon you are interested in and click a log icon from the displayed module detail sub-window.
  2. 'Runtime Log' window is displayed.
    • Image
  3. The following table is the explanation for each file.
    • #Name exampleDescription
      1DP00047852.demouser01.e300902.1A standard error log
      2DP00047852.demouser01.e300902.1.txtAn alias of the above file, which can open with many browsers at direct
      Remember that error causes is frequently described in this file when a module ends with an 'Abort' status.
      3DP00047852.demouser01.o300902.1A standard output log
      4DP00047852.demouser01.o300902.1.txtAn alias of the above file, which can open with many browsers at direct
      Remember that error causes is frequently described in this file when a module ends with an 'Abort' status.
      5DP00047852.demouser01.pe300902.1No meaning for users.
      6DP00047852.demouser01.pe300902.1.txtNo meaning for users.
      7DP00047852.demouser01.po300902.1No meaning for users.
      8DP00047852.demouser01.po300902.1.txtNo meaning for users.
      9evaluated.txtA log file about executed commands in one line format.
      10evaluated_command.txtA log file about executed commands in a human readable format.
      11input_checkA log file about the number of input files.
      12myscriptA local script to use only in this tool execution.
      13time.log.txtA log file about running time.
  4. A file example of DP00047852.demouser01.o300902.1.txt
    • Image
    • Contents of this file is totally depend on the executed tools.
    • Sometimes this file is empty.
  5. A file example of DP00047852.demouser01.e300902.1.txt
    • Image
    • Contents of this file is totally depend on the executed tools.
    • In many cases this file is empty when the tools correctly ended.
  6. A file example of evaluated_command.txt
    • Image
  7. A file example of time.log.txt
    • Image
    • In this file, real execution start time and execution end time are described.
    • Between the start time and the end time it does not contain system operation time and the waiting time for starting each tool.

More complex example


    • Image
    • A pipeline 'BWA (SE) + GE' constitute of 2 modules and total 5 tools.
  1. After open a 'Module Flow View', click a module about this pipeline and click a log icon from the displayed module detail sub-window.
    • Image
  2. 'Runtime Log' view about 'BWA (SE)' (the former module) which constitute of 3 tools.
    • Image
    • Image
    • Note that there are 3 sets of log files in 3 directories '1_C000000001', '2_C000000002' and '3_C000000003'.
    • Each of directories corresponds with the 3 tools 'bwa aln (single-end)', 'bwa samse (single-end)' and 'samToBam/sort/indexing SAMtools with head'.
    • Meaning of files in each log directory is same as described here.
  3. 'Runtime Log' view about 'loadBAM2ge_db (single-end)' (the latter module) which constitute of 2 modules.
    • Image
    • Note that there are 2 sets of log files in 2 directories '1_C000000001' and '2_C000000002'.
    • Each of directories corresponds with the 2 tools 'bam2ge_tsv (single-end)' and 'load_gbmap_data.pl (directional/non-directional)'.
    • Meaning of files in each log directory is same as described here.