Cwtch Server Monitoring Crashes Server on Raspberry Pi #136

New Issue

sarah · 2018-10-06T04:48:00Z

sarah commented

2018-10-06 04:48:00 +00:00

This might just be a Raspberry Pi / ARM thing, but we've been noticing our Raspberry Pi Cwtch Server deployment crashing on concurrent map writes in the monitoring software.

Considering we've been running CwtchServer for a while without issue in the monitoring this might be an issue without the PID API is implemented in the library, but we should look into this.

Error:

fatal error: concurrent map writes

goroutine 22 [running]:
runtime.throw(0x29c1b6, 0x15)
/usr/lib/go-1.7/src/runtime/panic.go:566 +0x78 fp=0x10632c58 sp=0x10632c4c
runtime.mapassign1(0x268668, 0x10668880, 0x10632cf8, 0x10632d4c)
/usr/lib/go-1.7/src/runtime/hashmap.go:458 +0x94 fp=0x10632cd8 sp=0x10632c58
github.com/struCoder/pidusage.stat(0x6c30, 0x298086, 0x4, 0x4, 0x0, 0x0)
go/src/github.com/struCoder/pidusage/pidusage.go:142 +0xc64 fp=0x10632ebc sp=0x10632cd8
github.com/struCoder/pidusage.wrapper.func1(0x6c30, 0x106688a0, 0x0, 0x0)
go/src/github.com/struCoder/pidusage/pidusage.go:40 +0x4c fp=0x10632ee0 sp=0x10632ebc
github.com/struCoder/pidusage.GetStat(0x6c30, 0x10632f10, 0x0, 0x0)
go/src/github.com/struCoder/pidusage/pidusage.go:152 +0x6c fp=0x10632ef8 sp=0x10632ee0
cwtch.im/cwtch/server/metrics.(*Monitors).Start.func3(0x1069ebb8, 0x1066a380)
go/src/cwtch.im/cwtch/server/metrics/monitors.go:37 +0x28 fp=0x10632f0c sp=0x10632ef8
cwtch.im/cwtch/server/metrics.(*monitorHistory).monitorThread(0x1069e800)
go/src/cwtch.im/cwtch/server/metrics/metrics.go:218 +0xfc fp=0x10632fcc sp=0x10632f0c
runtime.goexit()
/usr/lib/go-1.7/src/runtime/asm_arm.s:998 +0x4 fp=0x10632fcc sp=0x10632fcc
created by cwtch.im/cwtch/server/metrics.(*monitorHistory).Start

This *might* just be a Raspberry Pi / ARM thing, but we've been noticing our Raspberry Pi Cwtch Server deployment crashing on concurrent map writes in the monitoring software. Considering we've been running CwtchServer for a while without issue in the monitoring this might be an issue without the PID API is implemented in the library, but we should look into this. Error: fatal error: concurrent map writes goroutine 22 [running]: runtime.throw(0x29c1b6, 0x15) /usr/lib/go-1.7/src/runtime/panic.go:566 +0x78 fp=0x10632c58 sp=0x10632c4c runtime.mapassign1(0x268668, 0x10668880, 0x10632cf8, 0x10632d4c) /usr/lib/go-1.7/src/runtime/hashmap.go:458 +0x94 fp=0x10632cd8 sp=0x10632c58 github.com/struCoder/pidusage.stat(0x6c30, 0x298086, 0x4, 0x4, 0x0, 0x0) go/src/github.com/struCoder/pidusage/pidusage.go:142 +0xc64 fp=0x10632ebc sp=0x10632cd8 github.com/struCoder/pidusage.wrapper.func1(0x6c30, 0x106688a0, 0x0, 0x0) go/src/github.com/struCoder/pidusage/pidusage.go:40 +0x4c fp=0x10632ee0 sp=0x10632ebc github.com/struCoder/pidusage.GetStat(0x6c30, 0x10632f10, 0x0, 0x0) go/src/github.com/struCoder/pidusage/pidusage.go:152 +0x6c fp=0x10632ef8 sp=0x10632ee0 cwtch.im/cwtch/server/metrics.(*Monitors).Start.func3(0x1069ebb8, 0x1066a380) go/src/cwtch.im/cwtch/server/metrics/monitors.go:37 +0x28 fp=0x10632f0c sp=0x10632ef8 cwtch.im/cwtch/server/metrics.(*monitorHistory).monitorThread(0x1069e800) go/src/cwtch.im/cwtch/server/metrics/metrics.go:218 +0xfc fp=0x10632fcc sp=0x10632f0c runtime.goexit() /usr/lib/go-1.7/src/runtime/asm_arm.s:998 +0x4 fp=0x10632fcc sp=0x10632fcc created by cwtch.im/cwtch/server/metrics.(*monitorHistory).Start

dan commented

2018-11-21 21:44:37 +00:00

https://github.com/struCoder/pidusage/issues/8

https://github.com/struCoder/pidusage/pull/9

https://github.com/struCoder/pidusage/issues/8 https://github.com/struCoder/pidusage/pull/9

dan commented

2018-11-21 22:03:31 +00:00

#160