如何刪除emacs中的重複行

Question

我有一個很多行的文本，我的問題是如何刪除emacs中的重複行？在emacs或elisp軟件包中使用該命令，而無需外部utils。

例如：

this is line a 
this is line b 
this is line a 

刪除

this is line a 
this is line b 

Answer 1

將這個代碼到你的.emacs 3號線（同第1線）：

(defun uniq-lines (beg end) 
    "Unique lines in region. 
Called from a program, there are two arguments: 
BEG and END (region to sort)." 
    (interactive "r") 
    (save-excursion 
    (save-restriction 
     (narrow-to-region beg end) 
     (goto-char (point-min)) 
     (while (not (eobp)) 
     (kill-line 1) 
     (yank) 
     (let ((next-line (point))) 
      (while 
       (re-search-forward 
       (format "^%s" (regexp-quote (car kill-ring))) nil t) 
      (replace-match "" nil nil)) 
      (goto-char next-line)))))) 

用法：

M-x uniq-lines 

Answer 2

(defun unique-lines (start end) 
    "This will remove all duplicating lines in the region. 
Note empty lines count as duplicates of the empy line! All empy lines are 
removed sans the first one, which may be confusing!" 
    (interactive "r") 
    (let ((hash (make-hash-table :test #'equal)) (i -1)) 
    (dolist (s (split-string (buffer-substring-no-properties start end) "$" t) 
       (let ((lines (make-vector (1+ i) nil))) 
       (maphash 
        (lambda (key value) (setf (aref lines value) key)) 
        hash) 
       (kill-region start end) 
       (insert (mapconcat #'identity lines "\n")))) 
     (setq s       ; because Emacs can't properly 
             ; split lines :/ 
      (substring 
      s (position-if 
       (lambda (x) 
        (not (or (char-equal ?\n x) (char-equal ?\r x)))) s))) 
     (unless (gethash s hash) 
     (setf (gethash s hash) (incf i)))))) 

一種替代方案：

• 不會使用撤消記錄存儲匹配。
• 一般來說會更快（但如果你是在極限速度之後 - 建立一個前綴樹）。
• 具有替換所有以前的換行符的效果，無論它們是否與\n（UNIX樣式）一致。根據您的情況，這可能是獎金或劣勢。
• 如果您重新實現split-string以使其接受字符而不是正則表達式，您可以使它更好一點（更快）。

---

稍長，但是，也許，更多的有效的變體：

(defun split-string-chars (string chars &optional omit-nulls) 
    (let ((separators (make-hash-table)) 
     (last 0) 
     current 
     result) 
    (dolist (c chars) (setf (gethash c separators) t)) 
    (dotimes (i (length string) 
       (progn 
       (when (< last i) 
        (push (substring string last i) result)) 
       (reverse result))) 
     (setq current (aref string i)) 
     (when (gethash current separators) 
     (when (or (and (not omit-nulls) (= (1+ last) i)) 
        (/= last i)) 
      (push (substring string last i) result)) 
     (setq last (1+ i)))))) 

(defun unique-lines (start end) 
    "This will remove all duplicating lines in the region. 
Note empty lines count as duplicates of the empy line! All empy lines are 
removed sans the first one, which may be confusing!" 
    (interactive "r") 
    (let ((hash (make-hash-table :test #'equal)) (i -1)) 
    (dolist (s (split-string-chars 
       (buffer-substring-no-properties start end) '(?\n) t) 
       (let ((lines (make-vector (1+ i) nil))) 
       (maphash 
        (lambda (key value) (setf (aref lines value) key)) 
        hash) 
       (kill-region start end) 
       (insert (mapconcat #'identity lines "\n")))) 
     (unless (gethash s hash) 
     (setf (gethash s hash) (incf i)))))) 

Answer 3

在linux中，選擇區域，並鍵入

M-| uniq <RETURN> 

結果沒有重複在新的緩衝區。

Answer 4

如果你有Emacs 24.4或更新的版本，最簡單的方法是使用新的delete-duplicate-lines函數。需要注意的是

• 這個工作在一個區域，而不是一個緩衝區，所以選擇所需的文本第一
• 它保持原稿的相對順序，殺死副本

例如，如果你的輸入

test 
dup 
dup 
one 
two 
one 
three 
one 
test 
five 

M-x delete-duplicate-lines將使

test 
dup 
one 
two 
three 
five 

您可以選擇使用通用參數（C-u）作爲前綴搜索。結果將是

dup 
two 
three 
one 
test 
five 

信貸去emacsredux.com。

其他迂迴選項，不給完全一樣的結果，可以通過ESHELL：

1. sort -u;不保持原件的相對順序
2. uniq;更糟糕的是，它需要將其輸入進行排序