The scripts where created mainly to ease the work of the doc team. Two tasks are supported:
- Automate the work in case multiple content shall be "moved" from the DeveloperWiki to ReadTheDocs (c2m.sh)
- Analyse warning messages of the doc build process, identify related modules/projects/rst-files and ease the debug process (warnstats.sh)
They can be found in the doc repo in the doc/tools directory. Please check also Readme and header of both scripts for additional information.
Feel free to use them in your ONAP project to improve documentation ...
Of course ... your feedback is highly welcome.
CONTENT
c2m.sh
DESCRIPTION
c2m automates additional tasks required in case you want to export and convert a set of wiki pages. The export and first conversion to markdown is done by confluence2md, provided by viaboxx.
c2m processes a list of (to be exported) wiki pages, creates corresponding export directories, exports and converts pages (in various formats if required), opens an editor and cleans up afterwards. c2m checks also for problematic content in the export and creates a warning in case of detection.
USAGE
c2m.sh <your-page-list>
ISSUES
- markdown (md) output of confluence2md contains sometimes tags that are somehow "merged" with the topic headline; manual edit is required here
OPEN
- confluence2md does not support all of the currently used confluence page types (structured-macros) - result for unsupported pages is "not satisfying"; enhancements (java) are required
- opt: toc creation in root document in case you export a tree of documents to separate files
REQUIRED
- pandoc
- retext
- confluence2md
- java (older version for confluence2md)
ADDITIONAL FILES
example.pagelist: an example pagelist to demonstrate functionality
SEE ALSO
https://www.viaboxx.de/code/confluence2md/
https://github.com/viaboxxsystems/confluence2md
EXAMPLE PAGELIST
example pagelist (field descriptions below); it uses the delimiter "|" for the four fields per line
copy/paste page id and title from wiki; to get the wiki page_id you have to login to the wiki, open the page and choose e.g. the history
depth: use depth to follow down the child-pages hierarchy if required: -1=infinte, 0=no children, #=number of child-pages to follow
every hierarchy "0" entry will lead into the creation of a dedicated working directory where the page and child-pages are stored
for better readability you can add spaces to the list, but use "|" as a delimiter
lines starting with a # are filtered by c2m
hierarchy | page_id | page_title | depth
0 | 1018748 | ONAP Portal | 0
1.1 | 1018759 | ONAP Portal for users | 0
1.2 | 1018762 | ONAP Portal for administrators | 0
1.2.1 | 1018764 | Admins | 0
1.2.2 | 1018811 | Users | 0
1.2.3 | 1018821 | Portal Admins | 0
1.2.4 | 1018826 | Application Onboarding | 0
1.2.5 | 1018832 | Widget Onboarding | 0
1.2.6 | 1018835 | Edit Functional Menu | 0
1.2.7 | 16004953 | Portal Microservices Onboarding | 0
in case you want to export to only one single output page (that contains all child-pages of the above example) use:
0 | 1018748 | ONAP Portal | -1
warnstats.sh
DESCRIPTION
warnstat helps to find the onap modules (projects) and rst-files which are responsible for the most warnings during the documentation build process. It requires a tox build logfile, parses it line by line, prints out some statistics and provides links to the local rst file, its html version, the related link to readthedocs and as well the doc8 test result for the rst.
"rst html web doc8_(00000)" links can be opened easily by using mouse-over/context menu functionality of bash. files will be opened in your browser and rst editor!
USAGE
warnstatst.sh <your-tox-build-logfile>
NOTES
create the required tox logfile when you locally build the onap documentation:
tox | tee tox-build-logfile
ADDITIONAL FILES
- example-tox-build-log: an example tox logfile to demonstrate functionality
- created links in the warnstats output may not work if you use this example
EXAMPLE OUTPUT
(onapdocs) user@ubuntu:~/onapdocs/doc/tools$ ./warnstats.sh ../tox-build.log warnstats version 1.6.0 (2020-04-03) html build directory ..... /home/user/onapdocs/doc/docs/_build/html web base url ............. https://docs.onap.org/en/latest doc8 command ............. doc8 --verbose doc8 results directory ... /home/user/onapdocs/doc/tools/doc8_results tox logfile .............. ../tox-build.log 00656 LINES WITH WARNING IN FILE '../tox-build.log' ################################################################################ ~~~ MESSAGES LONG ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ################################################################################ 00656 WARNINGS IN TOTAL WITH 169 UNIQUE MESSAGES 00129 | document_isnt_included_in_any_toctree 00056 | Title_underline_too_short_ 00053 | Could_not_lex_literal_block_as_json__Highlighting_skipped_ 00035 | Unexpected_indentation_ 00029 | Block_quote_ends_without_a_blank_line_unexpected_unindent_ - - snip - - - - - - - - - - - - - - 00001 | Citation_Log_Architecture_is_not_referenced_ 00001 | Citation_Eventlet_is_not_referenced_ 00001 | Citation_Django_Document_is_not_referenced_ 00001 | Citation_django_deploy_is_not_referenced_ 00001 | Anonymous_hyperlink_mismatch_7_references_but_0_targets_ ################################################################################ ~~~ MESSAGES SHORTENED (FOR SIMPLE GROUPING) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ################################################################################ 00656 WARNINGS IN TOTAL WITH 46 UNIQUE MESSAGES 00129 | document_isnt_in 00094 | Could_not_lex_li 00072 | Duplicate_explic 00056 | Title_underline_ 00039 | Unknown_target_n - - snip - - - - - - - - - - - - - - 00001 | Citation_Log_Arc 00001 | Citation_Eventle 00001 | Citation_Django_ 00001 | Citation_django_ 00001 | Anonymous_hyperl ################################################################################ ~~~ MODULES ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ################################################################################ 00656 WARNINGS IN TOTAL IN 37 MODULES 00123 | dcaegen2.git 00078 | ccsdk 00043 | vnfrqts 00043 | multicloud 00039 | dmaap - - snip - - - - - - - - - - - - - - 00001 | vid.git 00001 | usecase-ui.git 00001 | so 00001 | logging-analytics 00001 | docs_use-cases ################################################################################ ~~~ MODULES WITH RSTFILES ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ################################################################################ 00656 WARNINGS IN TOTAL IN APPROX. 241 RST FILES aaf | index.rst | rst html web doc8_(00008) | 00006 aaf | README.rst | rst html web doc8_(00000) | 00001 aaf | service.rst | rst html web doc8_(00140) | 00002 aaf | usage.rst | rst html web doc8_(00007) | 00006 aai | AAIRESTAPI_AMSTERDAM.rst | rst html web doc8_(00008) | 00001 - - snip - - - - - - - - - - - - - - vnfrqts | VES_Registration_3_2.rst | rst html web doc8_(00076) | 00001 vnfsdk | csar-validation.rst | rst html web doc8_(00010) | 00002 vnfsdk | index.rst | rst html web doc8_(00003) | 00002 vnfsdk | release-notes.rst | rst html web doc8_(00009) | 00003 vnfsdk | VNFSDK-Marketplace-userguide-vendors.rst | rst html web doc8_(00002) | 00001 ################################################################################ ~~~ RSTFILES ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ################################################################################ 00656 WARNINGS IN TOTAL IN APPROX. 241 RST FILES 00030 | dmaap | Installation.rst | rst html web doc8_(00023) 00019 | ccsdk | workflow.rst | rst html web doc8_(00002) 00017 | dcaegen2.git | userguide.rst | rst html web doc8_(00051) 00016 | dcaegen2.git | mod-onboardingapi.rst | rst html web doc8_(00006) 00015 | appc.git | APPC_LCM_API_Guide.rst | rst html web doc8_(01121) - - snip - - - - - - - - - - - - - - 00001 | aai | bulkApi.rst | rst html web doc8_(00011) 00001 | aai | AAIRESTAPI_DUBLIN.rst | rst html web doc8_(00542) 00001 | aai | AAIRESTAPI_CASABLANCA.rst | rst html web doc8_(00535) 00001 | aai | AAIRESTAPI_AMSTERDAM.rst | rst html web doc8_(00008) 00001 | aaf | README.rst | rst html web doc8_(00000)