Hadoop 2.5.2 mvn構建失敗Windows 7 x64 - hadoop-common上的編譯ms-native-dll？

Question

我遵循Hadoop2OnWindows Wiki here，並且我多次閱讀源樹中的BUILDING.txt。我想我已經找到了所有的依賴關係，但是我得到了一個奇怪的錯誤（下面的完整日誌）Failed to execute goal (compile-ms-native-dll)。

我已經安裝了Windows SDK 7.1

下面的事情是在我的PATH：

• Cygwin/bin
• 的MSBuild（C:\Windows\Microsoft.NET\Framework64\v4.0.30319）
• 協議緩衝區2.5.0
• 的Zlib和zlib/include（以便它可以訪問zlib.h）
• Findbugs
• Maven的

我還設置Platform=x64，不用說，我JAVA_HOME和M2_HOME也設置。

我在Windows SDK 7.1命令提示符下運行mvn package -Pdist,native-win,docs -DskipTests -Dtar。

我早些時候有Gnuwin32這給了我下面的錯誤，但移動到Cygwin/bin懷疑可能是Gnuwin32的不妥之處，但我仍然得到以下錯誤。我該怎麼辦？

這裏的錯誤日誌，

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:exec (com 
pile-ms-native-dll) on project hadoop-common: Command execution failed. Process 
exited with an error: 1(Exit value: 1) -> [Help 1] 
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal o 
rg.codehaus.mojo:exec-maven-plugin:1.2:exec (compile-ms-native-dll) on project h 
adoop-common: Command execution failed. 
     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor 
.java:216) 
     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor 
.java:153) 
     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor 
.java:145) 
     at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProje 
ct(LifecycleModuleBuilder.java:116) 
     at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProje 
ct(LifecycleModuleBuilder.java:80) 
     at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThre 
adedBuilder.build(SingleThreadedBuilder.java:51) 
     at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(Lifecycl 
eStarter.java:120) 
     at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:347) 
     at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:154) 
     at org.apache.maven.cli.MavenCli.execute(MavenCli.java:582) 
     at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214) 
     at org.apache.maven.cli.MavenCli.main(MavenCli.java:158) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. 
java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces 
sorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Laun 
cher.java:289) 
     at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.jav 
a:229) 
     at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(La 
uncher.java:415) 
     at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java: 
356) 
Caused by: org.apache.maven.plugin.MojoExecutionException: Command execution fai 
led. 
     at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:345) 
     at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(Default 
BuildPluginManager.java:132) 
     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor 
.java:208) 
     ... 19 more 

這只是在hadoop-common，它的失敗。這裏有一些更多的日誌可能是有用的，我只是無法弄清楚它是什麼，雖然，

[INFO] 
[INFO] --- exec-maven-plugin:1.2:exec (compile-ms-native-dll) @ hadoop-common -- 
- 
[DEBUG] Configuring mojo org.codehaus.mojo:exec-maven-plugin:1.2:exec from plugi 
n realm ClassRealm[plugin>org.codehaus.mojo:exec-maven-plugin:1.2, parent: sun.m 
isc.Launcher$AppClassLoader@1e4ad14c] 
[DEBUG] Configuring mojo 'org.codehaus.mojo:exec-maven-plugin:1.2:exec' with bas 
ic configurator --> 
[DEBUG] (f) arguments = [C:\hdc\hadoop-common-project\hadoop-common/src/main/n 
ative/native.sln, /nologo, /p:Configuration=Release, /p:OutDir=C:\hdc\hadoop-com 
mon-project\hadoop-common\target/bin/, /p:CustomSnappyPrefix=, /p:CustomSnappyLi 
b=, /p:CustomSnappyInclude=, /p:RequireSnappy=false] 
[DEBUG] (f) basedir = C:\hdc\hadoop-common-project\hadoop-common 
[DEBUG] (f) classpathScope = runtime 
[DEBUG] (f) executable = msbuild 
[DEBUG] (f) longClasspath = false 
[DEBUG] (f) project = MavenProject: org.apache.hadoop:hadoop-common:2.5.2 @ C: 
\hdc\hadoop-common-project\hadoop-common\pom.xml 
[DEBUG] (f) session = org.apache.maven.execution.MavenSession@1e54fd3d 
[DEBUG] (f) skip = false 
[DEBUG] -- end configuration -- 
[DEBUG] Executing command line: msbuild C:\hdc\hadoop-common-project\hadoop-comm 
on/src/main/native/native.sln /nologo /p:Configuration=Release /p:OutDir=C:\hdc\ 
hadoop-common-project\hadoop-common\target/bin/ /p:CustomSnappyPrefix= /p:Custom 
SnappyLib= /p:CustomSnappyInclude= /p:RequireSnappy=false 
Building the projects in this solution one at a time. To enable parallel build, 
please add the "/m" switch. 
Build started 23-11-2014 00:07:43. 
Project "C:\hdc\hadoop-common-project\hadoop-common\src\main\native\native.sln" 
on node 1 (default targets). 
ValidateSolutionConfiguration: 
    Building solution configuration "Release|x64". 
Project "C:\hdc\hadoop-common-project\hadoop-common\src\main\native\native.sln" 
(1) is building "C:\hdc\hadoop-common-project\hadoop-common\src\main\native\nati 
ve.vcxproj" (2) on node 1 (default targets). 
InitializeBuildStatus: 
    Touching "..\..\..\target\native\Release\native.unsuccessfulbuild". 
ClCompile: 
    C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\amd64\CL.exe /c /I. 
.\winutils\include /I..\..\..\target\native\javah /IC:\Java\jdk1.7.0_71\include 
/IC:\Java\jdk1.7.0_71\include\win32 /I.\src /Zi /nologo /W3 /WX- /O2 /Oi /GL /D 
WIN32 /D NDEBUG /D _WINDOWS /D _USRDLL /D NATIVE_EXPORTS /D _WINDLL /D _UNICODE 
/D UNICODE /Gm- /EHsc /MD /GS /Gy /fp:precise /Zc:wchar_t /Zc:forScope /Fo"..\.. 
\..\target\native\Release\\" /Fd"..\..\..\target\native\Release\vcWindows7.1SDK. 
pdb" /Gd /TC /wd4244 /errorReport:queue src\org\apache\hadoop\io\compress\zlib\Z 
libCompressor.c src\org\apache\hadoop\io\compress\zlib\ZlibDecompressor.c 
    ZlibCompressor.c 
c:\hdc\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hadoop 
\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: C 
annot open include file: 'zlib.h': No such file or directory [C:\hdc\hadoop-comm 
on-project\hadoop-common\src\main\native\native.vcxproj] 
    ZlibDecompressor.c 
c:\hdc\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hadoop 
\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: C 
annot open include file: 'zlib.h': No such file or directory [C:\hdc\hadoop-comm 
on-project\hadoop-common\src\main\native\native.vcxproj] 
Done Building Project "C:\hdc\hadoop-common-project\hadoop-common\src\main\nativ 
e\native.vcxproj" (default targets) -- FAILED. 
Done Building Project "C:\hdc\hadoop-common-project\hadoop-common\src\main\nativ 
e\native.sln" (default targets) -- FAILED. 

Build FAILED. 

"C:\hdc\hadoop-common-project\hadoop-common\src\main\native\native.sln" (default 
target) (1) -> 
"C:\hdc\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj" (def 
ault target) (2) -> 
(ClCompile target) -> 
    c:\hdc\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hado 
op\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: 
Cannot open include file: 'zlib.h': No such file or directory [C:\hdc\hadoop-co 
mmon-project\hadoop-common\src\main\native\native.vcxproj] 
    c:\hdc\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hado 
op\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: 
Cannot open include file: 'zlib.h': No such file or directory [C:\hdc\hadoop-co 
mmon-project\hadoop-common\src\main\native\native.vcxproj] 

    0 Warning(s) 
    2 Error(s) 

Answer 1

從日誌記錄，顯然，你錯過zlib.h 你設置的環境變量ZLIB_HOME包含zlib.h目錄？ 注意把zlib.h目錄路徑是錯誤的，沒有必要的，不僅需要zlib的bin目錄，在大廈的導遊說https://svn.apache.org/viewvc/hadoop/common/branches/branch-2/BUILDING.txt?view=markup

此外，zlib.h需要一些頭不存在在Windows中，所以你需要下載這些頭文件，並將其放入與zlib.h相同的文件夾中。 有關詳細信息，請參閱Is there a replacement for unistd.h for Windows (Visual C)?得到unistd.h中， 和https://gist.github.com/ashelly/7776712得到getopt.h

對於那些鄉親在Win32（像我一樣），你可能會面臨更多的麻煩。關鍵的解決方案應該是編輯winutils和native package的.sln和.vcxprj文件，以便它們與Win32平臺兼容。

Answer 2

我個人使用的命令mvn clean package -Pdist,native-win -DskipTests -Dtar -Dmaven.javadoc.skip=true成功地在Windows 8版本構建2.4.1我有許多不同的問題，但不記得這是其中之一。但它似乎抱怨zlib。

Answer 3

下載的protobuf並將其添加到您的PATH

https://code.google.com/p/protobuf/downloads/detail?name=protoc-2.5.0-win32.zip&can=2&q=