好的，一天的實驗表明，無論如何，32位JVM都會以1 GB的IOException: Map failed出現故障。爲了規避在64位機器上映射的Size exceeds Integer.MAX_VALUE，應該使用多個可承受尺寸的緩衝器，例如，每個100 MB都很好。那是因爲buffers are addressed by integer。
什麼在這個問題上，可以同時保留所有的這些緩存在內存中打開的，即沒有必要關閉一個緩衝=>空你分配在未來有效地增加文件大小，如下面的演示證明
前至少在Windows中是
import Utils._, java.nio._, java.io._, java.nio.channels.FileChannel
object MmfDemo extends App {
val bufAddrWidth = 25 /*in bits*/ // Every element of the buff addresses a long
val BUF_SIZE_WORDS = 1 << bufAddrWidth ; val BUF_SIZE_BYTES = BUF_SIZE_WORDS << 3
val bufBitMask = BUF_SIZE_WORDS - 1
var buffers = Vector[LongBuffer]()
var capacity = 0 ; var pos = 0
def select(pos: Int) = {
val bufn = pos >> bufAddrWidth // higher bits of address denote the buffer number
//println(s"accessing $pos = " + (pos - buf * wordsPerBuf) + " in " + buf)
while (buffers.length <= bufn) expand
pass(buffers(bufn)){_.position(pos & bufBitMask)}
}
def get(address: Int = pos) = {
pos = address +1
select(address).get
}
def put(value: Long) {
//println("writing " + value + " to " + pos)
select(pos).put(value) ; pos += 1
}
def expand = {
val fromByte = buffers.length.toLong * BUF_SIZE_BYTES
println("adding " + buffers.length + "th buffer, total size expected " + format(fromByte + BUF_SIZE_BYTES) + " bytes")
// 32bit JVM: java.io.IOException: "Map failed" at the following line if buf size requested is larger than 512 mb
// 64bit JVM: IllegalArgumentException: Size exceeds Integer.MAX_VALUE
buffers :+= fc.map(FileChannel.MapMode.READ_WRITE, fromByte, BUF_SIZE_BYTES).asLongBuffer()
capacity += BUF_SIZE_WORDS
}
def rdAll(get: Int => Long) {
var firstMismatch = -1
val failures = (0 until parse(args(1))).foldLeft(0) { case(failures, i) =>
val got = get(i)
if (got != i && firstMismatch == -1) {firstMismatch = i; println("first mismatch at " +format(i) + ", value = " + format(got))}
failures + ?(got != i, 1, 0)
} ; println(format(failures) + " mismatches")
}
val raf = new RandomAccessFile("""C:\Temp\mmf""", "rw")
val fc = raf.getChannel
try {
if (args.length < 1) {
println ("usage1: buf_gen <len in long words>")
println ("usage1: raf_gen <len in long words>")
println("example: buf_gen 30m")
println("usage2: raf_rd <size in words>")
println("usage3: buf_rd <size in words>")
} else {
val t1 = System.currentTimeMillis
args(0) match {
case "buf_gen" => raf.setLength(0)
(0 until parse(args(1))) foreach {i => put(i.toLong)}
case "raf_gen" => raf.setLength(0)
(0 until parse(args(1))) foreach {i =>raf.writeLong(i.toLong)}
//fc.force(true)
case "rd_raf" => rdAll{i => raf.seek(i.toLong * 8) ; raf.readLong()}
case "rd_buf" => rdAll(get)
case u =>println("unknown command " + u)
} ; println("finished in " + (System.currentTimeMillis - t1) + " ms")
}
} finally {
raf.close ; fc.close
buffers = null ; System.gc /*GC needs to close the buffer*/}
}
object Utils {
val formatter = java.text.NumberFormat.getIntegerInstance
def format(l: Number) = formatter.format(l)
def ?[T](sel: Boolean, a: => T, b: => T) = if (sel) a else b
def parse(s: String) = {
val lc = s.toLowerCase()
lc.filter(_.isDigit).toInt *
?(lc.contains("k"), 1000, 1) *
?(lc.contains("m"), 1000*1000, 1)
}
def eqa[T](a: T, b: T) = assert(a == b, s"$a != $b")
def pass[T](a: T)(code: T => Unit) = {code(a) ; a}
}
。使用這個程序，我設法創建了比我的機器內存大的mmf文件（不是說JVM的-Xmx，在這些問題上根本不起作用）。只要減慢文件生成速度，用鼠標在Windows控制檯中選擇一些文本（程序會暫停，直到你釋放選擇），否則Windows將驅逐所有其他性能關鍵的工作人員到頁面文件，並且你的電腦將死於顛簸。
順便說一句，儘管我只寫入文件的末尾，但Windows可能會驅逐我未使用的千兆字節塊，但PC仍然死於抖動。另外，我注意到，我寫了塊實際上是讀
下輸出
adding 38th buffer, total size expected 12,480,000,000 bytes
adding 39th buffer, total size expected 12,800,000,000 bytes
伴隨着下面的系統要求
5:24,java,"QueryStandardInformationFile",mmf,"SUCCESS","AllocationSize: 12 480 000 000, EndOfFile: 12 480 000 000, NumberOfLinks: 1, DeletePending: False, Directory: False"
5:24,java,"SetEndOfFileInformationFile",mmf,"SUCCESS","EndOfFile: 12 800 000 000"
5:24,java,"SetAllocationInformationFile",mmf,"SUCCESS","AllocationSize: 12 800 000 000"
5:24,java,"CreateFileMapping",mmf,"FILE LOCKED WITH WRITERS","SyncType: SyncTypeCreateSection, PageProtection: "
5:24,java,"QueryStandardInformationFile",mmf,"SUCCESS","AllocationSize: 12 800 000 000, EndOfFile: 12 800 000 000, NumberOfLinks: 1, DeletePending: False, Directory: False"
5:24,java,"CreateFileMapping",mmf,"SUCCESS","SyncType: SyncTypeOther"
5:24,java,"ReadFile",mmf,"SUCCESS","Offset: 12 480 000 000, Length: 32 768, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal"
5:24,java,"ReadFile",mmf,"SUCCESS","Offset: 12 480 032 768, Length: 32 768, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal"
5:24,java,"ReadFile",mmf,"SUCCESS","Offset: 12 480 065 536, Length: 32 768, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal"
5:24,java,"ReadFile",mmf,"SUCCESS","Offset: 12 480 098 304, Length: 32 768, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal"
5:24,java,"ReadFile",mmf,"SUCCESS","Offset: 12 480 131 072, Length: 20 480, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal"
skipped 9000 reads
5:25,java,"ReadFile",mmf,"SUCCESS","Offset: 12 799 836 160, Length: 32 768, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal"
5:25,java,"ReadFile",mmf,"SUCCESS","Offset: 12 799 868 928, Length: 32 768, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal"
5:25,java,"ReadFile",mmf,"SUCCESS","Offset: 12 799 901 696, Length: 32 768, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal"
5:25,java,"ReadFile",mmf,"SUCCESS","Offset: 12 799 934 464, Length: 32 768, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal"
5:25,java,"ReadFile",mmf,"SUCCESS","Offset: 12 799 967 232, Length: 32 768, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal"
但那是另一回事。
事實證明，這個答案是Peter Lawrey's的重複，除了映射大緩衝區時我的問題專用於'映射失敗'和'整數範圍超過'，而原始問題與JVM中的OutOfMem有關，它無關與I/O。
這是什麼問題？ – the8472