LWP :: Simple＆LWP :: UserAgent不返回完整的網頁內容

Question

我有一個內部本地機器，託管分析，我需要處理。如果您擁有該網址，則會以JSON格式返回分析結果。例如，

http://1.2.3.4:8081/analytics/mydata/myNodeData?flat 

它會給JSON文件。現在我通過LWP :: Simple（也試過LWP :: UserAgent）將JSON文件放入一個變量中，然後解析它。它適用於我的大部分網址。問題在於，對於一​​些URL，它不會將整個數據讀入我的變量，但只是部分。

use LWP::Simple;    # From CPAN 
use LWP::UserAgent; 
use JSON qw(decode_json);  # From CPAN 
use Data::Dumper;    # Perl core module 
use Data::Diver qw{ Dive };   # for diving in the Hash 
use strict;      # Good practice 
use warnings;     # Good practice 

#using LWP Simple  
my $trendsurl = 'http://1.2.3.4:8081/analytics/mydata/myNodeData?flat'; 
my $json = get($trendsurl); 
die "Could not get $trendsurl!" unless defined $json; 
my $decoded_json = decode_json($json); 

#using LWP::UserAgent 
my $ua = LWP::UserAgent->new(); 
my $req = new HTTP::Request GET => $trendsurl; 
my $res = $ua->request($req); 
my $content = $res->content; 

如果這裏有用的是顯示調試信息的內容（decode_json）是一樣的。

DB<1> p $content 
{"NodeStatus": {"deleted": false, "disk_usage_info": [{"partition_space_availabl 
e_1k": 791475728, "partition_space_used_1k": 171611096, "partition_name": "/dev/ 
mapper/os-root", "partition_type": "ext4"}, {"partition_space_available_1k": 151 
200, "partition_space_used_1k": 39244, "partition_name": "/dev/vda3", "partition 
_type": "ext2"}], "process_info": [{"process_name": "XX-api:0", "process_s 
tate": "PROCESS_STATE_RUNNING", "last_stop_time": null, "start_count": 2, "core_ 
file_list": [], "last_start_time": "1461822985334246", "stop_count": 0, "last_ex 
it_time": null, "exit_count": 0}, {"process_name": "XX-config-nodemgr", "p 
rocess_state": "PROCESS_STATE_RUNNING", "last_stop_time": null, "start_count": 2 
, "core_file_list": [], "last_start_time": "1461822979324868", "stop_count": 0, 
"last_exit_time": null, "exit_count": 0}, {"process_name": "XX-discovery:0 
", "process_state": "PROCESS_STATE_RUNNING", "last_stop_time": null, "start_coun 
t": 2, "core_file_list": [], "last_start_time": "1461822983332516", "stop_count" 
: 0, "last_exit_time": null, "exit_count": 0}, {"process_name": "XX-svc-m 

    DB<2> 

你看到日誌文件被突然終止。如果我通過我的瀏覽器，它完全得到這個..

編輯：

添加以下eval語句而獲取URL，然後解碼JSON .. retreival沒有錯誤，但JSON解碼失敗，因爲它不是一個完整的JSON。

eval { $decoded_json = parse_json($json) }; 
if ($@) { 
    warn "func raised an exception: $@"; 
} 
func raised an exception: JSON error at line 1: Unexpected end of input parsing 
string starting from byte 1105 at Tester.pl line 28. 

編輯2： 我只是嘗試另一種方式......

#!/usr/bin/perl 

use LWP::UserAgent; 
use HTTP::Request; 

my $URL = 'http://1.2.3.4:8081/analytics/mydata/myNodeData?flat'; 

my $ua = LWP::UserAgent->new(); 
my $header = HTTP::Request->new(GET => $URL); 
my $request = HTTP::Request->new('GET', $URL, $header); 
my $response = $ua->request($request); 

if ($response->is_success){ 
    print "URL:$URL\nHeaders:\n"; 
    print $response->headers_as_string; 
}elsif ($response->is_error){ 
    print "Error:$URL\n"; 
    print $response->error_as_HTML; 
} 

和調試表明，$響應 - > is_success塊被選中，這裏是輸出。

URL:http://1.2.3.4:8081/analytics/mydata/myNodeData?flat 
Headers: 
Connection: close 
Date: Tue, 10 May 2016 19:49:40 GMT 
Content-Type: application/json 
Client-Aborted: die 
Client-Date: Tue, 10 May 2016 19:49:38 GMT 
Client-Peer: 1.2.3.4:8081 
Client-Response-Num: 1 
Client-Transfer-Encoding: chunked 
X-Died: read failed: An existing connection was forcibly closed by the remote ho 
st. at C:/Perl64/lib/LWP/Protocol/http.pm line 465. 

Answer 1

我使用此代碼或類似的東西，該代碼使用POST的參數，可以我得到一個大的JSON這個和它的作品以及

use LWP::UserAgent; 
use Data::Dumper; 
use JSON; 
use JSON::Parse; 

my $params = 
{(
'flat' => 1, 
)}; 

    my $server_endpoint = "http://1.2.3.4:8081/analytics/mydata/myNodeData"; 
    my $ua = LWP::UserAgent->new; 
    my $req = HTTP::Request->new(); 
    my $response = $ua->post($server_endpoint, $params); 
    my $result = $response->decoded_content(); 

    $result = from_json($result); 
    print Dumper $result;