'Broken pipe'異常是否會取消我的工作？

Question

目前，我正在使用144個TaskSlots在4臺機器的遠程集羣上運行Flink程序。 30分左右運行後，我收到以下錯誤：

INFO org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet - Info server for jobmanager: Failed to write json updates for job b2eaff8539c8c9b696826e69fb40ca14, because org.eclipse.jetty.io.RuntimeIOException: org.eclipse.jetty.io.EofException at org.eclipse.jetty.io.UncheckedPrintWriter.setError(UncheckedPrintWriter.java:107) at org.eclipse.jetty.io.UncheckedPrintWriter.write(UncheckedPrintWriter.java:280) at org.eclipse.jetty.io.UncheckedPrintWriter.write(UncheckedPrintWriter.java:295) at org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet.writeJsonUpdatesForJob(JobManagerInfoServlet.java:588) at org.apache.flink.runtime.jobmanager.web.JobManagerInfoServlet.doGet(JobManagerInfoServlet.java:209) at javax.servlet.http.HttpServlet.service(HttpServlet.java:734) at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:532) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:965) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:388) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:187) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:901) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:47) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:113) at org.eclipse.jetty.server.Server.handle(Server.java:352) at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:596) at org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:549) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:211) at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:425) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:489) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:436) at java.lang.Thread.run(Thread.java:745) Caused by: org.eclipse.jetty.io.EofException at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:905) at org.eclipse.jetty.http.AbstractGenerator.flush(AbstractGenerator.java:427) at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:78) at org.eclipse.jetty.server.HttpConnection$Output.flush(HttpConnection.java:1139) at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:159) at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:86) at java.io.ByteArrayOutputStream.writeTo(ByteArrayOutputStream.java:154) at org.eclipse.jetty.server.HttpWriter.write(HttpWriter.java:258) at org.eclipse.jetty.server.HttpWriter.write(HttpWriter.java:107) at org.eclipse.jetty.io.UncheckedPrintWriter.write(UncheckedPrintWriter.java:271) ... 24 more Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) at sun.nio.ch.IOUtil.write(IOUtil.java:51) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:470) at org.eclipse.jetty.io.nio.ChannelEndPoint.flush(ChannelEndPoint.java:185) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.flush(SelectChannelEndPoint.java:256) at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:849) ... 33 more

我知道java.io.IOException: Broken pipe意味着，JobManager失去了某種聯繫，所以我想整個作業失敗，我必須重新啓動它。儘管我認爲該過程不再運行，但WebInterface仍將其列爲正在運行。另外，當我使用jps來識別羣集上正在運行的進程時，JobManager仍然存在。所以我的問題是，如果我的工作丟失了，這個錯誤是否有時會發生，或者我的程序是否導致了這個錯誤。

編輯：我的任務管理器每隔幾秒仍然發送心跳，似乎正在運行。

Answer 1

這實際上是Flink的網絡服務器JobManagerInfoServlet的一個問題，由於java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method)的原因，無法將所請求作業的最新JSON更新發送到瀏覽器。因此，只有對服務器的GET請求失敗。

這樣的故障不應影響當前正在運行的Flink作業的執行。只需刷新瀏覽器（使用Flink的Web UI）應該發送另一個GET請求，然後希望成功完成。