CONTENT

`MOTIVATION`

The scripts where created mainly to ease the work of the doc team. Two tasks are supported:

Automate the work in case multiple content shall be "moved" from the DeveloperWiki to ReadTheDocs (c2m.sh)
Analyse warning messages of the doc build process, identify related modules/projects/rst-files and ease the debug process (warnstats.sh)

Both scripts can be found in the doc/tools directory contained in the doc repo. Please check also README.md and the header of both scripts for additional information.

Feel free to use them in your ONAP project to improve documentation ...

Of course ... your feedback is highly welcome.

`c2m.sh`

DESCRIPTION

c2m automates additional tasks required in case you want to export and convert a set of wiki pages. The export and first conversion to markdown is done by confluence2md, provided by viaboxx.

c2m processes a list of (to be exported) wiki pages, creates corresponding export directories, exports and converts pages (in various formats if required), opens an editor and cleans up afterwards. c2m checks also for problematic content in the export and creates a warning in case of detection.

USAGE

c2m.sh <your-page-list>

Before executing copy the confluence2md-2.1-fat.jar file from Maven repository to your working directory otherwise it won't work

ISSUES

markdown (md) output of confluence2md contains sometimes tags that are somehow "merged" with the topic headline; manual edit is required here

OPEN

confluence2md does not support all of the currently used confluence page types (structured-macros) - result for unsupported pages is "not satisfying"; enhancements (java) are required
workaround: copy the needed content to a new "blanc page" without using problematic macros and export it instead of the original page
opt: toc creation in root document in case you export a tree of documents to separate files

REQUIRED

pandoc
retext
confluence2md from https://repo1.maven.org/maven2/de/viaboxx/markdown/confluence2md/2.1/
java

ADDITIONAL FILES

example.pagelist: an example pagelist to demonstrate functionality

EXAMPLE PAGELIST

example pagelist (field descriptions below); it uses the delimiter "|" for the four fields per line
copy/paste page id and title from wiki; to get the wiki page_id you have to login to the wiki, open the page and choose e.g. the history
depth: use depth to follow down the child-pages hierarchy if required: -1=infinte, 0=no children, #=number of child-pages to follow
every hierarchy "0" entry will lead into the creation of a dedicated working directory where the page and child-pages are stored
for better readability you can add spaces to the list, but use "|" as a delimiter
lines starting with a # are filtered by c2m

# hierarchy | page_id  | page_title                      | depth

0           |  1018748 | ONAP Portal                     |  0
1.1         |  1018759 | ONAP Portal for users           |  0
1.2         |  1018762 | ONAP Portal for administrators  |  0
1.2.1       |  1018764 | Admins                          |  0
1.2.2       |  1018811 | Users                           |  0
1.2.3       |  1018821 | Portal Admins                   |  0
1.2.4       |  1018826 | Application Onboarding          |  0
1.2.5       |  1018832 | Widget Onboarding               |  0
1.2.6       |  1018835 | Edit Functional Menu            |  0
1.2.7       | 16004953 | Portal Microservices Onboarding |  0

in case you want to export to only one single output page (that contains all child-pages of the above example) use:

0           |  1018748 | ONAP Portal                     | -1

`warnstats.sh`

DESCRIPTION

warnstats helps to find the onap modules (projects) and rst-files which are responsible for the most warnings during the documentation build process. It requires a tox build logfile, parses it line by line, prints out some statistics and provides links to the local rst file, its html version, the related link to readthedocs and as well the doc8 test result for the rst.

"rst html web doc8_(00000)" links can be opened easily by using mouse-over/context menu functionality of bash. files will be opened in your browser and rst editor!

Number of warnings found in the logfile are shown as "nnnnn", e.g. "00129"

Number of doc8 warnings/errors in a rst file are shown as "doc8_(nnnnn)", e.g. "doc8_(00002)".

USAGE

warnstatst.sh <your-tox-build-logfile>

NOTES

create the required tox logfile when you locally build the onap documentation:

tox | tee tox-build-logfile

ADDITIONAL FILES

example-tox-build-log: an example tox logfile to demonstrate functionality (created links in the warnstats output may not work if you use this example)

EXAMPLE OUTPUT

(onapdocs) user@ubuntu:~/onapdocs/doc/tools$ ./warnstats.sh ../tox-build.log 

warnstats version 1.6.0 (2020-04-03)
html build directory ..... /home/user/onapdocs/doc/docs/_build/html
web base url ............. https://docs.onap.org/en/latest
doc8 command ............. doc8 --verbose
doc8 results directory ... /home/user/onapdocs/doc/tools/doc8_results
tox logfile .............. ../tox-build.log

00656 LINES WITH WARNING IN FILE '../tox-build.log'

################################################################################
~~~ MESSAGES LONG ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
################################################################################

00656 WARNINGS IN TOTAL WITH 169 UNIQUE MESSAGES
00129 | document_isnt_included_in_any_toctree
00056 | Title_underline_too_short_
00053 | Could_not_lex_literal_block_as_json__Highlighting_skipped_
00035 | Unexpected_indentation_
00029 | Block_quote_ends_without_a_blank_line_unexpected_unindent_
- - snip - - - - - - - - - - - - - -
00001 | Citation_Log_Architecture_is_not_referenced_
00001 | Citation_Eventlet_is_not_referenced_
00001 | Citation_Django_Document_is_not_referenced_
00001 | Citation_django_deploy_is_not_referenced_
00001 | Anonymous_hyperlink_mismatch_7_references_but_0_targets_

################################################################################
~~~ MESSAGES SHORTENED (FOR SIMPLE GROUPING) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
################################################################################

00656 WARNINGS IN TOTAL WITH 46 UNIQUE MESSAGES
00129 | document_isnt_in
00094 | Could_not_lex_li
00072 | Duplicate_explic
00056 | Title_underline_
00039 | Unknown_target_n
- - snip - - - - - - - - - - - - - -
00001 | Citation_Log_Arc
00001 | Citation_Eventle
00001 | Citation_Django_
00001 | Citation_django_
00001 | Anonymous_hyperl

################################################################################
~~~ MODULES ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
################################################################################

00656 WARNINGS IN TOTAL IN 37 MODULES
00123 | dcaegen2.git
00078 | ccsdk
00043 | vnfrqts
00043 | multicloud
00039 | dmaap
- - snip - - - - - - - - - - - - - -
00001 | vid.git
00001 | usecase-ui.git
00001 | so
00001 | logging-analytics
00001 | docs_use-cases

################################################################################
~~~ MODULES WITH RSTFILES ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
################################################################################

00656 WARNINGS IN TOTAL IN APPROX. 241 RST FILES
aaf     | index.rst                                | rst html web doc8_(00008) | 00006
aaf     | README.rst                               | rst html web doc8_(00000) | 00001
aaf     | service.rst                              | rst html web doc8_(00140) | 00002
aaf     | usage.rst                                | rst html web doc8_(00007) | 00006
aai     | AAIRESTAPI_AMSTERDAM.rst                 | rst html web doc8_(00008) | 00001
- - snip - - - - - - - - - - - - - -
vnfrqts | VES_Registration_3_2.rst                 | rst html web doc8_(00076) | 00001
vnfsdk  | csar-validation.rst                      | rst html web doc8_(00010) | 00002
vnfsdk  | index.rst                                | rst html web doc8_(00003) | 00002
vnfsdk  | release-notes.rst                        | rst html web doc8_(00009) | 00003
vnfsdk  | VNFSDK-Marketplace-userguide-vendors.rst | rst html web doc8_(00002) | 00001

################################################################################
~~~ RSTFILES ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
################################################################################

00656 WARNINGS IN TOTAL IN APPROX. 241 RST FILES
00030 | dmaap        | Installation.rst          | rst html web doc8_(00023)
00019 | ccsdk        | workflow.rst              | rst html web doc8_(00002)
00017 | dcaegen2.git | userguide.rst             | rst html web doc8_(00051)
00016 | dcaegen2.git | mod-onboardingapi.rst     | rst html web doc8_(00006)
00015 | appc.git     | APPC_LCM_API_Guide.rst    | rst html web doc8_(01121)
- - snip - - - - - - - - - - - - - -
00001 | aai          | bulkApi.rst               | rst html web doc8_(00011)
00001 | aai          | AAIRESTAPI_DUBLIN.rst     | rst html web doc8_(00542)
00001 | aai          | AAIRESTAPI_CASABLANCA.rst | rst html web doc8_(00535)
00001 | aai          | AAIRESTAPI_AMSTERDAM.rst  | rst html web doc8_(00008)
00001 | aaf          | README.rst                | rst html web doc8_(00000)

MOTIVATION

c2m.sh

DESCRIPTION

USAGE

ISSUES

OPEN

REQUIRED

ADDITIONAL FILES

SEE ALSO

EXAMPLE PAGELIST

warnstats.sh

DESCRIPTION

USAGE

NOTES

ADDITIONAL FILES

EXAMPLE OUTPUT

`MOTIVATION`

`c2m.sh`

`warnstats.sh`