從HTML頁面讀取固定寬度格式的文本表

我試圖從類似於以下的表格中讀取數據http://www.fec.gov/pubrec/fe1996/hraz.htm使用R但是一直無法取得進展。我意識到要做到這一點，我需要使用XML和RCurl，但儘管網絡上有許多關於類似問題的其他示例，但我無法解決這個問題。從HTML頁面讀取固定寬度格式的文本表

第一個問題是，該表只是一個表格查看時，但沒有編碼。把它作爲一個XML文檔處理我可以訪問表中的「數據」，但是因爲有幾個表我想得到我不認爲這是最優雅的解決方案。

將它作爲一個html文檔處理可能會更好，但我對xpathApply相對不熟悉，不知道如何獲取表中的實際「數據」，因爲它沒有被任何東西括起來（例如，i-/i或b-/b）。

我在過去使用xml文件取得了一些成功，但這是我第一次嘗試做類似於html文件的事情。特別是這些文件的結構似乎比其他我見過的例子少。

任何幫助，非常感謝。

來源

2013-04-17 johnson-shuffle

查看該頁面的來源，這不是一個表。 – A5C1D2H2I1M1N2O1R2T1

我想你想要xml閱讀器，而不是csv（表格）閱讀器 – Nishanth

再次查看頁面的源代碼，他們*嘗試*以固定寬度格式顯示數據，所以你可能只能「複製和粘貼「並使用'read.fwf'。然而。我提到他們已*嘗試*將數據固定爲寬度，但您可以看到「區域3」與其他區域的對齊方式不同，這將導入後需要進一步清理。 – A5C1D2H2I1M1N2O1R2T1

假設你可以閱讀html輸出到一個文本文件（複製+粘貼的等價形式Web瀏覽器），這應該讓你的存在方式的好大塊：

# x is the output from the website 


library(stringr) 
library(data.table) 

# First, remove commas from numbers (easiest to do at beginning) 
x <- gsub(",([0-9])", "\\1", x) 

# split the data by District 
districts <- strsplit(x, "DISTRICT *")[[1]] 

# separate out the header info 
headerInfo <- districts[[1]] 
districts <- tail(districts, -1) 


# grab the straggling district number, use it as a name and remove it 

    # end of first line 
    eofl <- str_locate(districts, "\n")[,2] 

    # trim white space and assign as name 
    names(districts) <- str_trim(substr(districts, 1, eofl)) 

    # remove first line 
    districts <- substr(districts, eofl+1, nchar(districts)) 

# replace the ending '-------' and trime white space 
    districts <- str_trim(str_replace_all(districts, "---*", "")) 

# Adjust delimeter (this is the tricky part) 

    ## more than two spaces are a spearator 
    districts <- str_replace_all(districts, " +", "\t") 

    ## lines that are total tallies are missing two columns. 
    ## thus, need to add two extra delims. After the first and third columns 

     # this function will 
     padDelims <- function(section, splton) { 
      # split into lines 
      section <- strsplit(section, splton)[[1]] 
      # identify lines starting with totals 
      LinesToFix <- str_detect(section, "^Total") 
      # pad appropriate columns 
      section[LinesToFix] <- sub("(.+)\t(.+)\t(.*)?", "\\1\t\t\\2\t\t\\3", section[LinesToFix]) 

      # any rows missing delims, pad at end 
      counts <- str_count(section, "\t") 
      toadd <- max(counts) - counts 
      section[ ] <- mapply(function(s, p) if (p==0) return (s) else paste0(s, paste0(rep("\t", p), collapse="")), section, toadd) 

      # paste it back together and return 
      paste(section, collapse=splton) 
     } 

    districts <- lapply(districts, padDelims, splton="\n") 

    # reading the table and simultaneously addding the district column 
    districtTables <- 
     lapply(names(districts), function(d) 
     data.table(read.table(text=districts[[d]], sep="\t"), district=d)) 
    # ... or without adding district number: 
    ##  lapply(districts, function(d) data.table(read.table(text=d, sep="\t"))) 

    # flatten it 
    votes <- do.call(rbind, districtTables) 
    setnames(votes, c("Candidate", "Party", "PrimVotes.Abs", "PrimVotes.Perc", "GeneralVotes.Abs", "GeneralVotes.Perc", "District"))

樣品表：

votes 

         Candidate  Party PrimVotes.Abs PrimVotes.Perc GeneralVotes.Abs GeneralVotes.Perc District 
1:     Salmon, Matt   R   33672   100.00  135634.00    60.18  1 
2:   Total Party Votes:     33672    NA    NA    NA  1 
3:              NA    NA    NA    NA  1 
4:      Cox, John  W(D)/D   1942   100.00   89738.00    39.82  1 
5:   Total Party Votes:      1942    NA    NA    NA  1 
6:              NA    NA    NA    NA  1 
7:   Total District Votes:     35614    NA  225372.00    NA  1 
8:     Pastor, Ed   D   29969   100.00   81982.00    65.01  2 
9:   Total Party Votes:     29969    NA    NA    NA  2 
10:              NA    NA    NA    NA  2 
... 
51:    Hayworth, J.D.   R   32554   100.00  121431.00    47.57  6 
52:   Total Party Votes:     32554    NA    NA    NA  6 
53:              NA    NA    NA    NA  6 
54:     Owens, Steve   D   35137   100.00  118957.00    46.60  6 
55:   Total Party Votes:     35137    NA    NA    NA  6 
56:              NA    NA    NA    NA  6 
57:    Anderson, Robert  LBT   148   100.00   14899.00    5.84  6 
58:              NA    NA    NA    NA  6 
59:   Total District Votes:     67839    NA  255287.00    NA  6 
60:              NA    NA    NA    NA  6 
61:   Total State Votes:     368185    NA  1356446.00    NA  6 
         Candidate  Party PrimVotes.Abs PrimVotes.Perc GeneralVotes.Abs GeneralVotes.Perc District

來源

2013-04-17 06:58:39

從HTML頁面讀取固定寬度格式的文本表

回答

相關問題