2018-10-11 09:47:36.610 INFO 2387 --- [main] o.o.d.collectors.datafile.MainApp : Starting MainApp on Chengkais-MacBook-Pro.local with PID 2387 (/Users/chengkaiyan/datafile/datafile-app-server/target/datafile-app-server.jar started by chengkaiyan in /Users/chengkaiyan/datafile/datafile-app-server) 2018-10-11 09:47:36.613 DEBUG 2387 --- [main] o.o.d.collectors.datafile.MainApp : Running with Spring Boot v2.0.4.RELEASE, Spring v5.0.5.RELEASE 2018-10-11 09:47:36.613 INFO 2387 --- [main] o.o.d.collectors.datafile.MainApp : The following profiles are active: prod 2018-10-11 09:47:39.671 TRACE 2387 --- [pool-2-thread-1] o.o.d.c.datafile.tasks.ScheduledTasks : Execution of tasks was registered 2018-10-11 09:47:39.672 INFO 2387 --- [parallel-1] o.o.d.c.d.c.EnvironmentProcessor : Loading configuration from system environment variables 2018-10-11 09:47:39.679 WARN 2387 --- [parallel-1] o.o.d.c.d.c.CloudConfiguration : Error in case of processing system environment, more details below: org.onap.dcaegen2.collectors.datafile.exceptions.EnvironmentLoaderException: $CONSUL_HOST environment has not been defined at org.onap.dcaegen2.collectors.datafile.configuration.EnvironmentProcessor.lambda$getConsulHost$0(EnvironmentProcessor.java:57) at java.util.Optional.orElseThrow(Optional.java:290) at org.onap.dcaegen2.collectors.datafile.configuration.EnvironmentProcessor.getConsulHost(EnvironmentProcessor.java:57) at org.onap.dcaegen2.collectors.datafile.configuration.EnvironmentProcessor.evaluate(EnvironmentProcessor.java:45) at org.onap.dcaegen2.collectors.datafile.configuration.CloudConfiguration.lambda$runTask$0(CloudConfiguration.java:64) at reactor.core.publisher.FluxDefer.subscribe(FluxDefer.java:47) at reactor.core.publisher.Flux.subscribe(Flux.java:6877) at reactor.core.publisher.FluxSubscribeOn$SubscribeOnSubscriber.run(FluxSubscribeOn.java:194) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2018-10-11 09:47:39.729 TRACE 2387 --- [pool-2-thread-1] o.o.d.c.d.tasks.DmaapConsumerTaskImpl : execute called with arg 2018-10-11 09:47:39.770 TRACE 2387 --- [pool-2-thread-1] o.o.d.c.d.s.DmaapReactiveWebClient : Request: GET http://localhost:2222/events/unauthenticated.VES_NOTIFICATION_OUTPUT/OpenDcae-c12/C12 2018-10-11 09:47:39.772 INFO 2387 --- [pool-2-thread-1] o.o.d.c.d.s.DmaapReactiveWebClient : Content-Type=application/json 2018-10-11 09:47:39.772 TRACE 2387 --- [pool-2-thread-1] o.o.d.c.d.s.DmaapReactiveWebClient : HTTP request headers: {Content-Type=[application/json]} 2018-10-11 09:47:39.778 TRACE 2387 --- [pool-2-thread-1] o.o.d.c.d.tasks.DmaapConsumerTaskImpl : consume called with arg MonoFlatMap 2018-10-11 09:47:40.076 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.d.s.DmaapReactiveWebClient : Response Status 200 2018-10-11 09:47:40.135 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.d.s.DmaapConsumerJsonParser : original message from message router: ["{\"event\":{\"commonEventHeader\":{\"startEpochMicrosec\":0000000000000,\"eventId\":\"FileReady_1797490e-10ae-4d48-9ea7-3d7d790b25e1\",\"timeZoneOffset\":\"UTC+05.30\",\"internalHeaderFields\":{\"collectorTimeStamp\":\"Wed, 09 19 2018 11:39:30 UTC\"},\"priority\":\"Normal\",\"version\":\"4.0.1\",\"reportingEntityName\":\"otenb5309\",\"sequence\":0,\"domain\":\"notification\",\"lastEpochMicrosec\":8745745764578,\"eventName\":\"Noti_RnNode-Ericsson_FileReady\",\"vesEventListenerVersion\":\"7.0.1\",\"sourceName\":\"oteNB5309\"},\"notificationFields\":{\"notificationFieldsVersion\":\"2.0\",\"changeType\":\"FileReady\",\"changeIdentifier\":\"PM_MEAS_FILES\",\"arrayOfNamedHashMap\":[{\"name\":\"fileFromFtps.tar.gz\",\"hashMap\":{\"location\":\"ftpes://myuser:mypass@localhost:21/fileFromFtps.tar.gz\",\"fileFormatType\":\"org.3GPP.32.435#measCollec\",\"fileFormatVersion\":\"V10\",\"compression\":\"gzip\"}}, {\"name\":\"fileFromSftp.tar.gz\",\"hashMap\":{\"location\":\"sftp://foo:pass@localhost:22/fileFromSftp.tar.gz\",\"fileFormatType\":\"org.3GPP.32.435#measCollec\",\"fileFormatVersion\":\"V10\",\"compression\":\"gzip\"}}]}}}"] 2018-10-11 09:47:40.137 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.d.s.DmaapConsumerJsonParser : starting to getJsonObjectFromAnArray! 2018-10-11 09:47:40.138 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.d.s.DmaapConsumerJsonParser : starting to getFileDataFromJson! 2018-10-11 09:47:40.143 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.d.s.DmaapConsumerJsonParser : starting to getFileDataFromJson! 2018-10-11 09:47:40.144 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.d.tasks.XnfCollectorTaskImpl : Entering execute with FileData{changeIdentifier=PM_MEAS_FILES, changeType=FileReady, name=fileFromFtps.tar.gz, location=ftpes://myuser:mypass@localhost:21/fileFromFtps.tar.gz, compression=gzip, fileFormatType=org.3GPP.32.435#measCollec, fileFormatVersion=V10} 2018-10-11 09:47:40.149 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.d.tasks.XnfCollectorTaskImpl : starting to collectFile 2018-10-11 09:47:40.150 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.datafile.ftp.FtpsClient : collectFile called with fileServerData: FileServerData{serverAddress=localhost, userId=myuser, password=mypass, port=21}, remoteFile: /fileFromFtps.tar.gz, localFile: target/fileFromFtps.tar.gz 2018-10-11 09:47:40.150 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.datafile.ftp.FtpsClient : retryCollectFile called 2018-10-11 09:47:40.531 INFO 2387 --- [reactor-http-nio-4] o.o.d.c.datafile.ftp.FtpsClient : complete setUpKeyManager 2018-10-11 09:47:40.537 INFO 2387 --- [reactor-http-nio-4] o.o.d.c.datafile.ftp.FtpsClient : complete setUpTrustedCA 2018-10-11 09:47:40.650 INFO 2387 --- [reactor-http-nio-4] o.o.d.c.datafile.ftp.FtpsClient : after ftp connect 2018-10-11 09:47:40.669 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.datafile.ftp.FtpsClient : setUpConnection return value: true 2018-10-11 09:47:40.670 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.datafile.ftp.FtpsClient : starting to getFile 2018-10-11 09:47:40.765 DEBUG 2387 --- [reactor-http-nio-4] o.o.d.c.datafile.ftp.FtpsClient : File target/fileFromFtps.tar.gz Download Successfull from xNF 2018-10-11 09:47:40.765 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.datafile.ftp.FtpsClient : starting to closeDownConnection 2018-10-11 09:47:40.769 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.datafile.ftp.FtpsClient : collectFile left with result: Download successful: true Error data: , errorData: 2018-10-11 09:47:40.771 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.d.tasks.XnfCollectorTaskImpl : Exiting execute with ConsumerDmaapModel{name=fileFromFtps.tar.gz, location=target/fileFromFtps.tar.gz, compression=gzip, fileFormatType=org.3GPP.32.435#measCollec, fileFormatVersion=V10} 2018-10-11 09:47:40.771 TRACE 2387 --- [reactor-http-nio-4] o.o.d.c.d.tasks.DmaapPublisherTaskImpl : Method called with arg ConsumerDmaapModel{name=fileFromFtps.tar.gz, location=target/fileFromFtps.tar.gz, compression=gzip, fileFormatType=org.3GPP.32.435#measCollec, fileFormatVersion=V10} 2018-10-11 09:47:40.773 TRACE 2387 --- [reactor-http-nio-4] .c.d.s.p.DmaapProducerReactiveHttpClient : Entering getDmaapProducerResponse with ConsumerDmaapModel{name=fileFromFtps.tar.gz, location=target/fileFromFtps.tar.gz, compression=gzip, fileFormatType=org.3GPP.32.435#measCollec, fileFormatVersion=V10} 2018-10-11 09:47:40.784 TRACE 2387 --- [reactor-http-nio-4] .c.d.s.p.DmaapProducerReactiveHttpClient : base64Creds...: ZHJhZG1pbjpkcmFkbWlu 2018-10-11 09:47:40.790 TRACE 2387 --- [reactor-http-nio-4] .c.d.s.p.DmaapProducerReactiveHttpClient : Starting to publish to DR 2018-10-11 09:47:41.019 DEBUG 2387 --- [reactor-http-nio-4] .c.d.w.RequestResponseLoggingInterceptor : ===========================request begin================================================ 2018-10-11 09:47:41.019 DEBUG 2387 --- [reactor-http-nio-4] .c.d.w.RequestResponseLoggingInterceptor : URI : https://localhost:3907/publish/1/fileFromFtps.tar.gz 2018-10-11 09:47:41.019 DEBUG 2387 --- [reactor-http-nio-4] .c.d.w.RequestResponseLoggingInterceptor : Method : PUT 2018-10-11 09:47:41.019 DEBUG 2387 --- [reactor-http-nio-4] .c.d.w.RequestResponseLoggingInterceptor : Headers : {Accept=[text/plain, application/json, application/*+json, */*], Content-Type=[application/octet-stream], X-ATT-DR-META=[{"compression":"gzip","fileFormatType":"org.3GPP.32.435#measCollec","fileFormatVersion":"V10"}], Authorization=[Basic ZHJhZG1pbjpkcmFkbWlu], Content-Length=[793]} 2018-10-11 09:47:41.020 DEBUG 2387 --- [reactor-http-nio-4] .c.d.w.RequestResponseLoggingInterceptor : Request body: ?H?[???ON-*?M+)(???.f?>000031Q??f?`???#ccCcc##cS3CSS???$?????????Y???UPYZs ^Q??C???3010?&&+?+D(@H?????8?A????e?tLbv4%?q???\??????"???T`dX?X??%?Gn!??@s?\n6gOW?C>?3{j28? U ??Vs2???&?d??%?%R?K?JR?3 ??y - ,???????2 ?D?:?6?GQFV??^`?v2562??:???^??Z?G]?????z?go??FqR?K?ym?,??~?????s,??9????>|:s?s??& ??}?M?[6?`???SG?5sg?8?S}3???0?&Af??+???<6O???J?r???L? ?h? tW?;??L??9?y&5?1|??a}8?)3?yi~Ym???_i?|??7.u?xg?cu??d?E?m?o??{?? W? ??????z?????bA?|[??????j??E????K?W????v???d's???6?000031Q??f?`???#ccCcc##cS3CSS???$?????????Y???UPYZs ^Q??C???3010?&&+?+D(@H?????8?A????e?tLbv4%?q???\??????"???T`dX?X??%?Gn!??@s?\n6gOW?C>?3{j28? U ??Vs2???&?d??%?%R?K?JR?3 ??y - ,???????2 ?D?:?6?GQFV??^`?v2562??:???^??Z?G]?????z?go??FqR?K?ym?,??~?????s,??9????>|:s?s??& ??}?M?[6?`???SG?5sg?8?S}3???0?&Af??+???<6O???J?r???L? ?h? tW?;??L??9?y&5?1|??a}8?)3?yi~Ym???_i?|??7.u?xg?cu??d?E?m?o??{?? W? ??????z?????bA?|[??????j??E????K?W????v???d's???6?000031Q??f?`???#ccCcc##cS3CSS???$?????????Y???UPYZs ^Q??C???3010?&&+?+D(@H?????8?A????e?tLbv4%?q???\??????"???T`dX?X??%?Gn!??@s?\n6gOW?C>?3{j28? U ??Vs2???&?d??%?%R?K?JR?3 ??y - ,???????2 ?D?:?6?GQFV??^`?v2562??:???^??Z?G]?????z?go??FqR?K?ym?,??~?????s,??9????>|:s?s??& ??}?M?[6?`???SG?5sg?8?S}3???0?&Af??+???<6O???J?r???L? ?h? tW?;??L??9?y&5?1|??a}8?)3?yi~Ym???_i?|??7.u?xg?cu??d?E?m?o??{?? W? ??????z?????bA?|[??????j??E????K?W????v???d's???6?000031Q??f?`???#ccCcc##cS3CSS???$?????????Y???UPYZs ^Q??C???3010?&&+?+D(@H?????8?A????e?tLbv4%?q???\??????"???T`dX?X??%?Gn!??@s?\n6gOW?C>?3{j28? U ??Vs2???&?d??%?%R?K?JR?3 ??y - ,???????2 ?D?:?6?GQFV??^`?v2562??:???^??Z?G]?????z?go??FqR?K?ym?,??~?????s,??9????>|:s?s??& ??}?M?[6?`???SG?5sg?8?S}3???0?&Af??+???<6O???J?r???L? ?h? tW?;??L??9?y&5?1|??a}8?)3?yi~Ym???_i?|??7.u?xg?cu??d?E?m?o??{?? W? ??????z?????bA?|[??????j??E????K?W????v???d's???6?000031Q??f?`???#ccCcc##cS3CSS???$?????????Y???UPYZs ^Q??C???3010?&&+?+D(@H?????8?A????e?tLbv4%?q???\??????"???T`dX?X??%?Gn!??@s?\n6gOW?C>?3{j28? U ??Vs2???&?d??%?%R?K?JR?3 ??y - ,???????2 ?D?:?6?GQFV??^`?v2562??:???^??Z?G]?????z?go??FqR?K?ym?,??~?????s,??9????>|:s?s??& ??}?M?[6?`???SG?5sg?8?S}3???0?&Af??+???<6O???J?r???L? ?h? tW?;??L??9?y&5?1|??a}8?)3?yi~Ym???_i?|??7.u?xg?cu??d?E?m?o??{?? W? ??????z?????bA?|[??????j??E????K?W????v???d's???6?000031Q??f?`???#ccCcc##cS3CSS???$?????????Y???UPYZs ^Q??C???3010?&&+?+D(@H?????8?A????e?tLbv4%?q???\??????"???T`dX?X??%?Gn!??@s?\n6gOW?C>?3{j28? U ??Vs2???&?d??%?%R?K?JR?3 ??y - ,???????2 ?D?:?6?GQFV??^`?v2562??:???^??Z?G]?????z?go??FqR?K?ym?,??~?????s,??9????>|:s?s??& ??}?M?[6?`???SG?5sg?8?S}3???0?&Af??+???<6O???J?r???L? ?h? tW?;??L??9?y&5?1|??a}8?)3?yi~Ym???_i?|??7.u?xg?cu??d?E?m?o??{?? W? ??????z?????bA?|[??????j??E????K?W????v???d's???6?