您现在的位置是:首页 > 学习笔记 > Go语言Go语言
Go语言TCP Socket通信编程(转)
冰山2019-05-10【Go语言】人已围观
简介一、模型
二、TCP连接的建立
三、Socket读写
四、Socket属性
五、关闭连接
Golang的主要 设计目标之一就是面向大规模后端服务程序,网络通信这块是服务端 程序必不可少也是至关重要的一部分。在日常应用中,我们也可以看到Go中的net以及其subdirectories下的包均是“高频+刚需”,而TCP socket则是网络编程的主流,即便您没有直接使用到net中有关TCP Socket方面的接口,但net/http总是用到了吧,http底层依旧是用tcp socket实现的。
网络编程方面,我们最常用的就是tcp socket编程了,在posix标准出来后,socket在各大主流OS平台上都得到了很好的支持。关于tcp programming,最好的资料莫过于W. Richard Stevens 的网络编程圣经《UNIX网络 编程 卷1:套接字联网API》 了,书中关于tcp socket接口的各种使用、行为模式、异常处理讲解的十分细致。Go是自带runtime的跨平台编程语言,Go中暴露给语言使用者的tcp socket api是建立OS原生tcp socket接口之上的。由于Go runtime调度的需要,golang tcp socket接口在行为特点与异常处理方面与OS原生接口有着一些差别。这篇博文的目标就是整理出关于Go tcp socket在各个场景下的使用方法、行为特点以及注意事项。
一、模型
从tcp socket诞生后,网络编程架构模型也几经演化,大致是:“每进程一个连接” –> “每线程一个连接” –> “Non-Block + I/O多路复用(linux epoll/windows iocp/freebsd darwin kqueue/solaris Event Port)”。伴随着模型的演化,服务程序愈加强大,可以支持更多的连接,获得更好的处理性能。
目前主流web server一般均采用的都是”Non-Block + I/O多路复用”(有的也结合了多线程、多进程)。不过I/O多路复用也给使用者带来了不小的复杂度,以至于后续出现了许多高性能的I/O多路复用框架, 比如libevent、libev、libuv等,以帮助开发者简化开发复杂性,降低心智负担。不过Go的设计者似乎认为I/O多路复用的这种通过回调机制割裂控制流 的方式依旧复杂,且有悖于“一般逻辑”设计,为此Go语言将该“复杂性”隐藏在Runtime中了:Go开发者无需关注socket是否是 non-block的,也无需亲自注册文件描述符的回调,只需在每个连接对应的goroutine中以“block I/O”的方式对待socket处理即可,这可以说大大降低了开发人员的心智负担。一个典型的Go server端程序大致如下:
//go-tcpsock/server.go func handleConn(c net.Conn) { defer c.Close() for { // read from the connection // ... ... // write to the connection //... ... } } func main() { l, err := net.Listen("tcp", ":8888") if err != nil { fmt.Println("listen error:", err) return } for { c, err := l.Accept() if err != nil { fmt.Println("accept error:", err) break } // start a new goroutine to handle // the new connection. go handleConn(c) } }
二、TCP连接的建立
conn, err := net.Dial("tcp", "google.com:80") if err != nil { //handle error } // read or write on conn
//go-tcpsock/conn_establish/client1.go ... ... func main() { log.Println("begin dial...") conn, err := net.Dial("tcp", ":8888") if err != nil { log.Println("dial error:", err) return } defer conn.Close() log.Println("dial ok") }
$go run client1.go 2015/11/16 14:37:41 begin dial... 2015/11/16 14:37:41 dial error: dial tcp :8888: getsockopt: connection refused
2、对方服务的listen backlog满
//go-tcpsock/conn_establish/server2.go ... ... func main() { l, err := net.Listen("tcp", ":8888") if err != nil { log.Println("error listen:", err) return } defer l.Close() log.Println("listen ok") var i int for { time.Sleep(time.Second * 10) if _, err := l.Accept(); err != nil { log.Println("accept error:", err) break } i++ log.Printf("%d: accept a new connection\n", i) } }
//go-tcpsock/conn_establish/client2.go ... ... func establishConn(i int) net.Conn { conn, err := net.Dial("tcp", ":8888") if err != nil { log.Printf("%d: dial error: %s", i, err) return nil } log.Println(i, ":connect to server ok") return conn } func main() { var sl []net.Conn for i := 1; i < 1000; i++ { conn := establishConn(i) if conn != nil { sl = append(sl, conn) } } time.Sleep(time.Second * 10000) }
$go run server2.go 2015/11/16 21:55:41 listen ok 2015/11/16 21:55:51 1: accept a new connection 2015/11/16 21:56:01 2: accept a new connection ... ... $go run client2.go 2015/11/16 21:55:44 1 :connect to server ok 2015/11/16 21:55:44 2 :connect to server ok 2015/11/16 21:55:44 3 :connect to server ok ... ... 2015/11/16 21:55:44 126 :connect to server ok 2015/11/16 21:55:44 127 :connect to server ok 2015/11/16 21:55:44 128 :connect to server ok 2015/11/16 21:55:52 129 :connect to server ok 2015/11/16 21:56:03 130 :connect to server ok 2015/11/16 21:56:14 131 :connect to server ok ... ...
$sysctl -a|grep kern.ipc.somaxconn kern.ipc.somaxconn: 128
如果我在ubuntu 14.04上运行上述server程序,我们的client端初始可以成功建立499条连接。
2015/11/16 22:03:31 128 :connect to server ok 2015/11/16 22:04:48 129: dial error: dial tcp :8888: getsockopt: operation timed out
3、网络延迟较大,Dial阻塞并超时
在连接建立阶段,多数情况下,Dial是可以满足需求的,即便阻塞一小会儿。但对于某些程序而言,需要有严格的连接时间限定,如果一定时间内没能成功建立连接,程序可能会需要执行一段“异常”处理逻辑,为此我们就需要DialTimeout了。下面的例子将Dial的最长阻塞时间限制在2s内,超出这个时长,Dial将返回timeout error:
//go-tcpsock/conn_establish/client3.go ... ... func main() { log.Println("begin dial...") conn, err := net.DialTimeout("tcp", "104.236.176.96:80", 2*time.Second) if err != nil { log.Println("dial error:", err) return } defer conn.Close() log.Println("dial ok") }
执行结果如下(需要模拟一个延迟较大的网络环境):
$go run client3.go 2015/11/17 09:28:34 begin dial... 2015/11/17 09:28:36 dial error: dial tcp 104.236.176.96:80: i/o timeout
三、Socket读写
连接建立起来后,我们就要在conn上进行读写,以完成业务逻辑。前面说过Go runtime隐藏了I/O多路复用的复杂性。语言使用者只需采用goroutine+Block I/O的模式即可满足大部分场景需求。Dial成功后,方法返回一个net.Conn接口类型变量值,这个接口变量的动态类型为一个*TCPConn:
//$GOROOT/src/net/tcpsock_posix.go type TCPConn struct { conn }
//$GOROOT/src/net/net.go type conn struct { fd *netFD } func (c *conn) ok() bool { return c != nil && c.fd != nil } // Implementation of the Conn interface. // Read implements the Conn Read method. func (c *conn) Read(b []byte) (int, error) { if !c.ok() { return 0, syscall.EINVAL } n, err := c.fd.Read(b) if err != nil && err != io.EOF { err = &OpError{Op: "read", Net: c.fd.net, Source: c.fd.laddr, Addr: c.fd.raddr, Err: err} } return n, err } // Write implements the Conn Write method. func (c *conn) Write(b []byte) (int, error) { if !c.ok() { return 0, syscall.EINVAL } n, err := c.fd.Write(b) if err != nil { err = &OpError{Op: "write", Net: c.fd.net, Source: c.fd.laddr, Addr: c.fd.raddr, Err: err} } return n, err }
//go-tcpsock/read_write/client2.go ... ... func main() { if len(os.Args) <= 1 { fmt.Println("usage: go run client2.go YOUR_CONTENT") return } log.Println("begin dial...") conn, err := net.Dial("tcp", ":8888") if err != nil { log.Println("dial error:", err) return } defer conn.Close() log.Println("dial ok") time.Sleep(time.Second * 2) data := os.Args[1] conn.Write([]byte(data)) time.Sleep(time.Second * 10000) }
//go-tcpsock/read_write/server2.go ... ... func handleConn(c net.Conn) { defer c.Close() for { // read from the connection var buf = make([]byte, 10) log.Println("start to read from conn") n, err := c.Read(buf) if err != nil { log.Println("conn read error:", err) return } log.Printf("read %d bytes, content is %s\n", n, string(buf[:n])) } } ... ...
$go run client2.go hi 2015/11/17 13:30:53 begin dial... 2015/11/17 13:30:53 dial ok $go run server2.go 2015/11/17 13:33:45 accept a new connection 2015/11/17 13:33:45 start to read from conn 2015/11/17 13:33:47 read 2 bytes, content is hi ...
3、Socket中有足够数据
$go run client2.go abcdefghij12345 2015/11/17 13:38:00 begin dial... 2015/11/17 13:38:00 dial ok $go run server2.go 2015/11/17 13:38:00 accept a new connection 2015/11/17 13:38:00 start to read from conn 2015/11/17 13:38:02 read 10 bytes, content is abcdefghij 2015/11/17 13:38:02 start to read from conn 2015/11/17 13:38:02 read 5 bytes, content is 12345
4、Socket关闭
“有数据关闭”是指在client关闭时,socket中还有server端未读取的数据,我们在go-tcpsock/read_write/client3.go和server3.go中模拟这种情况
$go run client3.go hello 2015/11/17 13:50:57 begin dial... 2015/11/17 13:50:57 dial ok $go run server3.go 2015/11/17 13:50:57 accept a new connection 2015/11/17 13:51:07 start to read from conn 2015/11/17 13:51:07 read 5 bytes, content is hello 2015/11/17 13:51:17 start to read from conn 2015/11/17 13:51:17 conn read error: EOF
5、读取操作超时
//go-tcpsock/read_write/client4.go ... ... func main() { log.Println("begin dial...") conn, err := net.Dial("tcp", ":8888") if err != nil { log.Println("dial error:", err) return } defer conn.Close() log.Println("dial ok") data := make([]byte, 65536) conn.Write(data) time.Sleep(time.Second * 10000) } //go-tcpsock/read_write/server4.go ... ... func handleConn(c net.Conn) { defer c.Close() for { // read from the connection time.Sleep(10 * time.Second) var buf = make([]byte, 65536) log.Println("start to read from conn") c.SetReadDeadline(time.Now().Add(time.Microsecond * 10)) n, err := c.Read(buf) if err != nil { log.Printf("conn read %d bytes, error: %s", n, err) if nerr, ok := err.(net.Error); ok && nerr.Timeout() { continue } return } log.Printf("read %d bytes, content is %s\n", n, string(buf[:n])) } }
$go run server4.go 2015/11/17 14:21:17 accept a new connection 2015/11/17 14:21:27 start to read from conn 2015/11/17 14:21:27 conn read 0 bytes, error: read tcp 127.0.0.1:8888->127.0.0.1:60970: i/o timeout 2015/11/17 14:21:37 start to read from conn 2015/11/17 14:21:37 read 65536 bytes, content is
1、成功写
前面例子着重于Read,client端在Write时并未判断Write的返回值。所谓“成功写”指的就是Write调用返回的n与预期要写入的数据长度相等,且error = nil。这是我们在调用Write时遇到的最常见的情形,这里不再举例了。
2、写阻塞
TCP连接通信两端的OS都会为该连接保留数据缓冲,一端调用Write后,实际上数据是写入到OS的协议栈的数据缓冲的。TCP是全双工通信,因此每个方向都有独立的数据缓冲。当发送方将对方的接收缓冲区以及自身的发送缓冲区写满后,Write就会阻塞。我们来看一个例子:client5.go和server.go。
//go-tcpsock/read_write/client5.go ... ... func main() { log.Println("begin dial...") conn, err := net.Dial("tcp", ":8888") if err != nil { log.Println("dial error:", err) return } defer conn.Close() log.Println("dial ok") data := make([]byte, 65536) var total int for { n, err := conn.Write(data) if err != nil { total += n log.Printf("write %d bytes, error:%s\n", n, err) break } total += n log.Printf("write %d bytes this time, %d bytes in total\n", n, total) } log.Printf("write %d bytes in total\n", total) time.Sleep(time.Second * 10000) } //go-tcpsock/read_write/server5.go ... ... func handleConn(c net.Conn) { defer c.Close() time.Sleep(time.Second * 10) for { // read from the connection time.Sleep(5 * time.Second) var buf = make([]byte, 60000) log.Println("start to read from conn") n, err := c.Read(buf) if err != nil { log.Printf("conn read %d bytes, error: %s", n, err) if nerr, ok := err.(net.Error); ok && nerr.Timeout() { continue } } log.Printf("read %d bytes, content is %s\n", n, string(buf[:n])) } } ... ...
$go run client5.go 2015/11/17 14:57:33 begin dial... 2015/11/17 14:57:33 dial ok 2015/11/17 14:57:33 write 65536 bytes this time, 65536 bytes in total 2015/11/17 14:57:33 write 65536 bytes this time, 131072 bytes in total 2015/11/17 14:57:33 write 65536 bytes this time, 196608 bytes in total 2015/11/17 14:57:33 write 65536 bytes this time, 262144 bytes in total 2015/11/17 14:57:33 write 65536 bytes this time, 327680 bytes in total 2015/11/17 14:57:33 write 65536 bytes this time, 393216 bytes in total 2015/11/17 14:57:33 write 65536 bytes this time, 458752 bytes in total 2015/11/17 14:57:33 write 65536 bytes this time, 524288 bytes in total 2015/11/17 14:57:33 write 65536 bytes this time, 589824 bytes in total 2015/11/17 14:57:33 write 65536 bytes this time, 655360 bytes in total
$go run server5.go 2015/11/17 15:07:01 accept a new connection 2015/11/17 15:07:16 start to read from conn 2015/11/17 15:07:16 read 60000 bytes, content is 2015/11/17 15:07:21 start to read from conn 2015/11/17 15:07:21 read 60000 bytes, content is 2015/11/17 15:07:26 start to read from conn 2015/11/17 15:07:26 read 60000 bytes, content is .... client端: 2015/11/17 15:07:01 write 65536 bytes this time, 720896 bytes in total 2015/11/17 15:07:06 write 65536 bytes this time, 786432 bytes in total 2015/11/17 15:07:16 write 65536 bytes this time, 851968 bytes in total 2015/11/17 15:07:16 write 65536 bytes this time, 917504 bytes in total 2015/11/17 15:07:27 write 65536 bytes this time, 983040 bytes in total 2015/11/17 15:07:27 write 65536 bytes this time, 1048576 bytes in total .... ...
3、写入部分数据
Write操作存在写入部分数据的情况,比如上面例子中,当client端输出日志停留在“write 65536 bytes this time, 655360 bytes in total”时,我们杀掉server5,这时我们会看到client5输出以下日志:
... 2015/11/17 15:19:14 write 65536 bytes this time, 655360 bytes in total 2015/11/17 15:19:16 write 24108 bytes, error:write tcp 127.0.0.1:62245->127.0.0.1:8888: write: broken pipe 2015/11/17 15:19:16 write 679468 bytes in total
显然Write并非在655360这个地方阻塞的,而是后续又写入24108后发生了阻塞,server端socket关闭后,我们看到Wrote返回er != nil且n = 24108,程序需要对这部分写入的24108字节做特定处理。
4、写入超时
如果非要给Write增加一个期限,那我们可以调用SetWriteDeadline方法。我们copy一份client5.go,形成client6.go,在client6.go的Write之前增加一行timeout设置代码:
conn.SetWriteDeadline(time.Now().Add(time.Microsecond * 10))
$go run client6.go 2015/11/17 15:26:34 begin dial... 2015/11/17 15:26:34 dial ok 2015/11/17 15:26:34 write 65536 bytes this time, 65536 bytes in total ... ... 2015/11/17 15:26:34 write 65536 bytes this time, 655360 bytes in total 2015/11/17 15:26:34 write 24108 bytes, error:write tcp 127.0.0.1:62325->127.0.0.1:8888: i/o timeout 2015/11/17 15:26:34 write 679468 bytes in total
Goroutine safe
type conn struct { fd *netFD }
// Network file descriptor. type netFD struct { // locking/lifetime of sysfd + serialize access to Read and Write methods fdmu fdMutex // immutable until Close sysfd int family int sotype int isConnected bool net string laddr Addr raddr Addr // wait server pd pollDesc }
func (fd *netFD) Read(p []byte) (n int, err error) { if err := fd.readLock(); err != nil { return 0, err } defer fd.readUnlock() if err := fd.pd.PrepareRead(); err != nil { return 0, err } for { n, err = syscall.Read(fd.sysfd, p) if err != nil { n = 0 if err == syscall.EAGAIN { if err = fd.pd.WaitRead(); err == nil { continue } } } err = fd.eofError(n, err) break } if _, ok := err.(syscall.Errno); ok { err = os.NewSyscallError("read", err) } return } func (fd *netFD) Write(p []byte) (nn int, err error) { if err := fd.writeLock(); err != nil { return 0, err } defer fd.writeUnlock() if err := fd.pd.PrepareWrite(); err != nil { return 0, err } for { var n int n, err = syscall.Write(fd.sysfd, p[nn:]) if n > 0 { nn += n } if nn == len(p) { break } if err == syscall.EAGAIN { if err = fd.pd.WaitWrite(); err == nil { continue } } if err != nil { break } if n == 0 { err = io.ErrUnexpectedEOF break } } if _, ok := err.(syscall.Errno); ok { err = os.NewSyscallError("write", err) } return nn, err }
四、Socket属性
原生Socket API提供了丰富的sockopt设置接口,但Golang有自己的网络架构模型,golang提供的socket options接口也是基于上述模型的必要的属性设置。包括
- SetKeepAlive
- SetKeepAlivePeriod
- SetLinger
- SetNoDelay (默认no delay)
- SetWriteBuffer
- SetReadBuffer
不过上面的Method是TCPConn的,而不是Conn的,要使用上面的Method的,需要type assertion:
tcpConn, ok := c.(*TCPConn) if !ok { //error handle } tcpConn.SetNoDelay(true)
五、关闭连接
和前面的方法相比,关闭连接算是最简单的操作了。由于socket是全双工的,client和server端在己方已关闭的socket和对方关闭的socket上操作的结果有不同。看下面例子:
//go-tcpsock/conn_close/client1.go ... ... func main() { log.Println("begin dial...") conn, err := net.Dial("tcp", ":8888") if err != nil { log.Println("dial error:", err) return } conn.Close() log.Println("close ok") var buf = make([]byte, 32) n, err := conn.Read(buf) if err != nil { log.Println("read error:", err) } else { log.Printf("read % bytes, content is %s\n", n, string(buf[:n])) } n, err = conn.Write(buf) if err != nil { log.Println("write error:", err) } else { log.Printf("write % bytes, content is %s\n", n, string(buf[:n])) } time.Sleep(time.Second * 1000) } //go-tcpsock/conn_close/server1.go ... ... func handleConn(c net.Conn) { defer c.Close() // read from the connection var buf = make([]byte, 10) log.Println("start to read from conn") n, err := c.Read(buf) if err != nil { log.Println("conn read error:", err) } else { log.Printf("read %d bytes, content is %s\n", n, string(buf[:n])) } n, err = c.Write(buf) if err != nil { log.Println("conn write error:", err) } else { log.Printf("write %d bytes, content is %s\n", n, string(buf[:n])) } } ... ...
上述例子的执行结果如下:
$go run server1.go 2015/11/17 17:00:51 accept a new connection 2015/11/17 17:00:51 start to read from conn 2015/11/17 17:00:51 conn read error: EOF 2015/11/17 17:00:51 write 10 bytes, content is $go run client1.go 2015/11/17 17:00:51 begin dial... 2015/11/17 17:00:51 close ok 2015/11/17 17:00:51 read error: read tcp 127.0.0.1:64195->127.0.0.1:8888: use of closed network connection 2015/11/17 17:00:51 write error: write tcp 127.0.0.1:64195->127.0.0.1:8888: use of closed network connection
微信扫一扫~
支付宝扫一扫~
很赞哦! ()
上一篇:go操作数据库之gorm
下一篇:Git常用基本用法
相关文章
随机图文
Go语言TCP Socket通信编程(转)
一、模型 二、TCP连接的建立 三、Socket读写 四、Socket属性 五、关闭连接变量
1.变量 2.变量类型 3.变量声明 4.变量初始化 5.变量赋值 6.自动推导类型 7.多重赋值匿名变量 8.数据置换 9.变量命名规范Golang之网络编程
1.协议 1.1协议 1.2典型协议 2.层与协议 2.1分层方式 3.通信过程 4.Socket编程 5.网络应用程序设计模式 5.1模式介绍 5.2优缺点 6.TCP通信 C/S模型通信 7.UDP通信 C/S模型通信 8.TCP通信并发C/S模型通信 9.UDP通信并发C/S模型通信 10.UDP与TCP的差异Go并发
1.并发 1.1并发概述 1.2进程并发 1.3线程并发 1.4协程并发 1.5Go并发 2.Goroutine 2.1Goroutine定义 2.2Goroutine的创建 2.3Goroutine特性 3.channel管道 3.1定义channel变量 3.2无缓冲的channel 3.3有缓冲的channel 3.4关闭channel 4.读写锁与互斥锁 4.1死锁 4.2互斥锁 4.3读写锁