Cygnus-NGSI將不會保存PostgreSQL中的數據

我試圖用cygnus-ngsi將某些數據保存在postgreSQL數據庫中，但沒有任何反應。我使用docker-compose在Docker容器中運行所有服務。Cygnus-NGSI將不會保存PostgreSQL中的數據

搬運工-compose.yml：

... 
cygnus: 
image: fiware/cygnus-ngsi:latest 
hostname: cygnus 
container_name: cygnus_fiware 
volumes: 
    - ./config/cygnus/cygnus.conf:/opt/apache-flume/conf/agent.conf 
    - ./config/cygnus/grouping_rules.conf:/opt/apache-flume/conf/grouping_rules.conf 
links: 
    - orion 
    - postgres 
expose: 
    - "5050" 
ports: 
    - "5050:5050" 

postgres: 
restart: always 
image: postgres:latest 
container_name: postgres_fiware 
volumes: 
    - ./data/db/postgres:/var/lib/postgresql/data 
ports: 
    - "5432:5432" 
expose: 
    - "5432" 
environment: 
    - POSTGRES_USER=teste 
    - POSTGRES_DB=newdb 
    - POSTGRES_PASSWORD=123456789

agent.conf從搬運工-撰寫

cygnus-ngsi.sources = http-source 
cygnus-ngsi.sinks = postgresql-sink 
cygnus-ngsi.channels = postgresql-channel 

#============================================= 
# source configuration 
# channel name where to write the notification events 
cygnus-ngsi.sources.http-source.channels = postgresql-channel 
# source class, must not be changed 
cygnus-ngsi.sources.http-source.type = org.apache.flume.source.http.HTTPSource 
# listening port the Flume source will use for receiving incoming notifications 
cygnus-ngsi.sources.http-source.port = 5050 
# Flume handler that will parse the notifications, must not be changed 
cygnus-ngsi.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.NGSIRestHandler 
# URL target 
cygnus-ngsi.sources.http-source.handler.notification_target = /notify 
# default service (service semantic depends on the persistence sink) 
cygnus-ngsi.sources.http-source.handler.default_service = default 
# default service path (service path semantic depends on the persistence sink) 
cygnus-ngsi.sources.http-source.handler.default_service_path =/
# source interceptors, do not change 
cygnus-ngsi.sources.http-source.interceptors = ts gi 
# TimestampInterceptor, do not change 
cygnus-ngsi.sources.http-source.interceptors.ts.type = timestamp 
# GroupingInterceptor, do not change 
cygnus-ngsi.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor$Builder 
# Grouping rules for the GroupingInterceptor, put the right absolute path to the file if necessary 
# see the doc/design/interceptors document for more details 
cygnus-ngsi.sources.http-source.interceptors.gi.grouping_rules_conf_file = /opt/apache-flume/conf/grouping_rules.conf 

# ============================================ 
# NGSIPostgreSQLSink configuration 
# channel name from where to read notification events 
cygnus-ngsi.sinks.postgresql-sink.channel = postgresql-channel 
# sink class, must not be changed 
cygnus-ngsi.sinks.postgresql-sink.type = com.telefonica.iot.cygnus.sinks.NGSIPostgreSQLSink 
# true applies the new encoding, false applies the old encoding. 
cygnus-ngsi.sinks.postgresql-sink.enable_encoding = false 
# true if name mappings are enabled for this sink, false otherwise 
cygnus-ngsi.sinks.postgresql-sink.enable_name_mappings = false 
# true if the grouping feature is enabled for this sink, false otherwise 
cygnus-ngsi.sinks.postgresql-sink.enable_grouping = false 
# true if lower case is wanted to forced in all the element names, false otherwise 
cygnus-ngsi.sinks.postgresql-sink.enable_lowercase = false 
# the FQDN/IP address where the PostgreSQL server runs 
cygnus-ngsi.sinks.postgresql-sink.postgresql_host = postgres 
# the port where the PostgreSQL server listens for incomming connections 
cygnus-ngsi.sinks.postgresql-sink.postgresql_port = 5432 
# the name of the postgresql database 
cygnus-ngsi.sinks.postgresql-sink.postgresql_database = newdb 
# a valid user in the PostgreSQL server 
cygnus-ngsi.sinks.postgresql-sink.postgresql_username = teste 
# password for the user above 
cygnus-ngsi.sinks.postgresql-sink.postgresql_password = 123456789 
# how the attributes are stored, either per row either per column (row, column) 
cygnus-ngsi.sinks.postgresql-sink.attr_persistence = row 
# select the data_model: dm-by-service-path or dm-by-entity 
cygnus-ngsi.sinks.postgresql-sink.data_model = dm-by-entity 
# number of notifications to be included within a processing batch 
cygnus-ngsi.sinks.postgresql-sink.batch_size = 100 
# timeout for batch accumulation 
cygnus-ngsi.sinks.postgresql-sink.batch_timeout = 30 
# number of retries upon persistence error 
cygnus-ngsi.sinks.postgresql-sink.batch_ttl = 10 




# ============================================= 
# postgresql-channel configuration 
# channel type (must not be changed) 
cygnus-ngsi.channels.postgresql-channel.type = memory 
# capacity of the channel 
cygnus-ngsi.channels.postgresql-channel.capacity = 1000 
# amount of bytes that can be sent per transaction 
cygnus-ngsi.channels.postgresql-channel.transactionCapacity = 100

消息：

cygnus_fiware | time=2017-06-19T15:11:15.132Z | lvl=WARN | corr=4af15a0c-5501-11e7-aa0a-0242ac130004 | trans=508576db-1443-4c64-bfc9-629d1a0b250e | srv=espometeo | subsrv=/environment | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[257] : [NGSIRestHandler] Unnecessary header 
cygnus_fiware | time=2017-06-19T15:11:15.133Z | lvl=INFO | corr=89ac4c66-5501-11e7-850f-0242ac130004 | trans=3ae0dc99-de51-49a3-937f-42d887b7e7d7 | srv=espometeo | subsrv=/environment | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[282] : [NGSIRestHandler] Starting internal transaction (3ae0dc99-de51-49a3-937f-42d887b7e7d7) 
cygnus_fiware | time=2017-06-19T15:11:15.133Z | lvl=INFO | corr=89ac4c66-5501-11e7-850f-0242ac130004 | trans=3ae0dc99-de51-49a3-937f-42d887b7e7d7 | srv=espometeo | subsrv=/environment | comp=cygnus-ngsi | op=getEvents | msg=com.telefonica.iot.cygnus.handlers.NGSIRestHandler[299] : [NGSIRestHandler] Received data ({ "subscriptionId" : "5947d328e143997a02b11008", "originator" : "localhost", "contextResponses" : [ {  "contextElement" : {  "type" : "EstacaoMeteo",  "isPattern" : "false",  "id" : "Estacao3",  "attributes" : [   {   "name" : "Humidity",   "type" : "float",   "value" : "35.3",   "metadatas" : [    {    "name" : "TimeInstant",    "type" : "ISO8601",    "value" : "2017-06-24T13:03:00"    }   ]   },   {   "name" : "Temperature",   "type" : "float",   "value" : "15.2",   "metadatas" : [    {    "name" : "TimeInstant",    "type" : "ISO8601",    "value" : "2017-06-24T13:03:00"    }   ]   }  ]  },  "statusCode" : {  "code" : "200",  "reasonPhrase" : "OK"  } } ]}) 



cygnus_fiware | time=2017-06-19T15:11:36.141Z | lvl=INFO | corr=89ac4c66-5501-11e7-850f-0242ac130004 | trans=3ae0dc99-de51-49a3-937f-42d887b7e7d7 | srv=espometeo | subsrv=/environment | comp=cygnus-ngsi | op=persistAggregation | msg=com.telefonica.iot.cygnus.sinks.NGSIPostgreSQLSink[479] : [postgresql-sink] Persisting data at NGSIPostgreSQLSink. Schema (espometeo), Table (environment_estacao3_estacaometeo), Fields ((recvTimeTs,recvTime,fiwareServicePath,entityId,entityType,attrName,attrType,attrValue,attrMd)), Values (('1497885075139','2017-06-19T15:11:15.139Z','/environment','Estacao3','EstacaoMeteo','Humidity','float','35.3','[{"name":"TimeInstant","type":"ISO8601","value":"2017-06-24T13:03:00"}]'),('1497885075139','2017-06-19T15:11:15.139Z','/environment','Estacao3','EstacaoMeteo','Temperature','float','15.2','[{"name":"TimeInstant","type":"ISO8601","value":"2017-06-24T13:03:00"}]')) 
cygnus_fiware | time=2017-06-19T15:11:36.142Z | lvl=WARN | corr=89ac4c66-5501-11e7-850f-0242ac130004 | trans=3ae0dc99-de51-49a3-937f-42d887b7e7d7 | srv=espometeo | subsrv=/environment | comp=cygnus-ngsi | op=processNewBatches | msg=com.telefonica.iot.cygnus.sinks.NGSISink[541] :

好像天鵝越來越從所有數據獵戶座和正確的，但是當我去postgresql數據庫，沒有什麼。有人已經有這個問題？

持久性錯誤消息：

cygnus_fiware | time=2017-06-22T09:45:06.092Z | lvl=ERROR | corr=N/A | trans=N/A | srv=N/A | subsrv=N/A | comp=cygnus-ngsi | op=processRollbackedBatches | msg=com.telefonica.iot.cygnus.sinks.NGSISink[398] : CygnusPersistenceError. -, null. Stack trace: [com.telefonica.iot.cygnus.sinks.NGSIPostgreSQLSink.persistAggregation(NGSIPostgreSQLSink.java:504), com.telefonica.iot.cygnus.sinks.NGSIPostgreSQLSink.persistBatch(NGSIPostgreSQLSink.java:231), com.telefonica.iot.cygnus.sinks.NGSISink.processRollbackedBatches(NGSISink.java:390), com.telefonica.iot.cygnus.sinks.NGSISink.process(NGSISink.java:372), org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68), org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147), java.lang.Thread.run(Thread.java:748)]

天鵝啓動誤差：

cygnus_fiware | + exec /usr/lib/jvm/java-6-sun/bin/java -Xms512m -Xmx1g -Dcom.sun.management.jmxremote -Dflume.root.logger=INFO,console -Duser.timezone=UTC -Dfile.encoding=UTF-8 -cp '/opt/apache-flume/conf:/opt/apache-flume/lib/*:/opt/apache-flume/plugins.d/cygnus/lib/*:/opt/apache-flume/plugins.d/cygnus/libext/*' -Djava.library.path= com.telefonica.iot.cygnus.nodes.CygnusApplication -f /opt/apache-flume/conf/agent.conf -n cygnus-ngsi -p 8081 
cygnus_fiware | /opt/apache-flume/bin/cygnus-flume-ng: line 232: /usr/lib/jvm/java-6-sun/bin/java: No such file or directory 
cygnus_fiware | /opt/apache-flume/bin/cygnus-flume-ng: line 232: exec: /usr/lib/jvm/java-6-sun/bin/java: cannot execute: No such file or directory

來源

2017-06-20 Naff16

您是否看到espometeo架構的內容？ 'environment_estacao3_estacaometeo'表怎麼樣？是否有與整個過程有關的錯誤？ – Dalton

我看到的唯一錯誤是JAVA_HOME未設置，但我無法修復它。由於數據庫爲空，我看不到espometeo模式。我已經嘗試創建架構和表，但沒有任何反應。 – Naff16

從[Cygnus文檔]（http://fiware-cygnus.readthedocs.io/en/1.2.2/cygnus-ngsi/flume_extensions_catalogue/ngsi_postgresql_sink/）： 'PostgreSQL數據庫命名約定在使用PostgreSQL之前您需要創建要使用的數據庫。必須說PostgreSQL只接受字母數字字符和下劃線（_）。在編寫表名時，所有其他字符將被轉義爲下劃線（_）。「你是對的嗎？你是否已經創建了模式和表？ – Dalton

我通過在agent.conf文件中將false更改爲true來解決該問題：cygnus-ngsi.sinks.postgresql-sink.backend.enable_cache = true。

來源

2017-07-05 15:57:52 Naff16

有PostgreSQL的問題水槽時，不啓用緩存。它被描述在this問題上。

來源

2017-06-30 06:27:36 frb

謝謝，我已經賣掉了以前的評論n的問題。我應該發佈它作爲最後的評論。 – Naff16

Cygnus-NGSI將不會保存PostgreSQL中的數據

回答

相關問題