IRC log for #asterisk on 20160107

00:08.56*** join/#asterisk rrittgarn (~rrittgarn@75-150-221-196-Illinois.hfc.comcastbusiness.net)
00:13.49Bordrany out there have any advice on where to look? getting this every few seconds to minutes on all extensions across different model phones and ata's
00:13.51Bordr[2016-01-06 02:32:09] NOTICE[2692] chan_sip.c: Peer '111' is now Lagged. (2038ms / 2000ms)
00:15.16WIMPyYou have a network or a load issue.
00:16.35Bordrthats what i was thinking... top shows .04 on the high side, no io wait, etc... network can iperf at 98% of Gbps
00:17.30Bordrserver is qc with 8gb of ram on ssd raid1
00:17.58WIMPyIt can be internal to chan_sip. It can be triggered by resolver issues.
00:18.35Bordrassuming you mean dns? this is all local lan, no router/nat between and all direct IP without dns
00:18.53WIMPyAlso is you have lots of peers with qualify enabled, make sure to use the option to not do them all at once.
00:18.56Bordrironically the external extensions (4 of them work fine)
00:19.03WIMPyYes, DNS.
00:19.43WIMPystarts to sound interesting.
00:19.49Bordronly 20 or so peers on this server
00:20.04Bordrqualify is default setting, phone register every 1hr
00:20.23Bordrspent the day swapping hardware and tracing wires... no luck
00:20.34WIMPyThat surely does not sound like load.
00:20.39Bordrthe only other time i have seen anything similar was due to switching loop
00:21.20*** part/#asterisk kharwell (kharwell@nat/digium/x-ppbddvebnangkiqn)
00:21.32Bordrfor diag i swapped out 48p poe switch with a 8p/4 poe trendnet pos... same problem so I'm thinking it's on the server.
00:46.30*** join/#asterisk C1ph3r (~C1ph3r@179.187.169.73.dynamic.adsl.gvt.net.br)
00:51.20*** join/#asterisk F2Knight (~F2Knight@c-50-139-86-39.hsd1.or.comcast.net)
00:55.34youtmonis there a free softy for asterisk that will filter based on country of origin to avoid scam and spams to network ports?
00:56.47WIMPyThere are DNS implementaions, but the quality of their results is a little questionable.
00:59.20*** join/#asterisk fstd (~fstd@unaffiliated/fisted)
01:03.44Bordryoutmon - you may want to block that traffic at a firewall level
01:04.17BordrWIMPy - Do you have any other suggestions that I can try? or maybe any other thoughts as to where I can look for troubleshooting?
01:04.59WIMPyFind the networks of the ISP your customers use and allow those.
01:18.16*** join/#asterisk DivideBy0 (~DivideBy0@unaffiliated/divideby0x0)
01:27.35*** join/#asterisk Tenhi_ (~tenhi@static-ip-69-64-50-196.inaddr.ip-pool.com)
01:31.19*** join/#asterisk Bordr (~quassel@23-24-156-49-static.hfc.comcastbusiness.net)
01:38.54*** join/#asterisk jasonwert (~wert@71.89.137.28)
01:46.36*** join/#asterisk Bordr (~quassel@23-24-156-49-static.hfc.comcastbusiness.net)
01:51.54*** join/#asterisk Bordr (~Bordr@23-24-156-49-static.hfc.comcastbusiness.net)
02:19.52*** join/#asterisk jasonwert (~wert@71.89.137.28)
02:25.43*** join/#asterisk coppice (~chatzilla@123203240102.ctinets.com)
03:02.41*** join/#asterisk simplydrew (~simplydre@unaffiliated/simplydrew)
03:10.54Bordrhave a problem still with:
03:10.55Bordr[2016-01-06 20:10:16] NOTICE[2662] chan_sip.c: Peer '117' is now UNREACHABLE!  Last qualify: 14
03:10.55Bordr[2016-01-06 20:10:26] NOTICE[2662] chan_sip.c: Peer '117' is now Reachable. (16ms / 2000ms)
03:11.52BordrI have tried everything I can think of to troubleshoot. I have checked server load, network stability, etc. If I ping the IP's of the phones that this occurs on there are no issues. I mean ping from the server to the ip of the phone shows no delays or issues yet asterisk believes there is.
03:12.50Bordranyone out there have any suggestions as to where I can look to find the cause of the issue? check messages, asterisk/full with no luck
03:13.23saint_what type of peer is it ?
03:13.50Bordrmost are aastra 6867i phones and couple are SPA112 ata's. All on the local subnet.
03:14.03saint_but THIS one
03:14.08saint_117
03:14.32BordrThe problem occurs to several extensions 101-119 all have the problem and are aastra 6867is
03:15.00BordrThe only phones that don't have this issue are offsite and go over wan.
03:15.32youtmonwhats the round trip latency when you ping one of the phones?
03:16.13saint_I don t know how much debug you did, but if you say it is most likely with th eaastra 6867i , I would switch an extension between aastra 6867i and SPA112 and see if the problem follows
03:16.46Bordrthe problem follows both but only when they are on lan, over wan they both work fine.
03:17.03Bordrlatency is between 0.734ms and 3ms
03:17.07saint_only lan ?
03:17.17saint_are they all on the same data switch ?
03:17.20Bordryes only the lan has the issue not the wan connected phones
03:17.27Bordryes they are now. replaced the switch today
03:17.37saint_what brand switch is it ?
03:17.59Bordrthe new one is a temporary "trendnet" poe switch (non-managed)
03:18.20Bordrremoved the zyxel GS-1948P today in an effort to trouble shoot
03:18.47saint_do they all become unreachable at the same time ?
03:18.53saint_or is it random ?
03:18.55Bordrno, at random
03:18.56saint_or every 10 minutes ?
03:19.04Bordrin bursts is more like it
03:19.18Bordrsometime multiple times a minute others like now every 5-15 minutes
03:19.27saint_are you SURE when you reboot asterisk / switch for example the first time they go off line is totally different from the first time they went off line before you restarted ?
03:19.53Bordrthis is a live tail -f /var/log/asterisk/full:
03:19.55Bordr[2016-01-06 20:04:12] NOTICE[2662] chan_sip.c: Peer '117' is now Reachable. (16ms / 2000ms)
03:19.55Bordr[2016-01-06 20:04:14] NOTICE[2662] chan_sip.c: Peer '125' is now Reachable. (49ms / 2000ms)
03:19.55Bordr[2016-01-06 20:10:16] NOTICE[2662] chan_sip.c: Peer '117' is now UNREACHABLE!  Last qualify: 14
03:19.55Bordr[2016-01-06 20:10:26] NOTICE[2662] chan_sip.c: Peer '117' is now Reachable. (16ms / 2000ms)
03:19.55Bordr[2016-01-06 20:19:19] NOTICE[2662] chan_sip.c: Peer '125' is now UNREACHABLE!  Last qualify: 46
03:19.58Bordr[2016-01-06 20:19:30] NOTICE[2662] chan_sip.c: Peer '125' is now Reachable. (1048ms / 2000ms)
03:20.34Bordrserver was physically rebooted 20 minutes ago
03:20.45saint_unreacheable means what it means. something in your network is fucked up. how many phones do you have connected to this server ?
03:21.00Bordrright now for testing 3 phones and 1 ata
03:21.12Bordr2 softphones and single aastra for testing are offsite
03:21.26saint_have you tried port mirroring on the switch, while running tcpdump on asterisk ?
03:21.46Bordrthe problem is * says unreachable but i can ping from the pbx to the phone the entire time without any change in ms
03:21.58*** join/#asterisk spicyramen (~Adium@c-50-184-3-185.hsd1.ca.comcast.net)
03:22.17Bordrran a large tcpdump (over 2 hours) to a txt file and reviewed, but found nothing abnormal
03:22.33saint_ha. so maybe the phone is no good then (binary wise) - but i think Asterisk test its peer reacheability through pings - not sure though
03:23.18Bordrcould agree with that, have tried diff firmware on each phone and the problem extends to the ATAs SPA112 as well
03:23.35Bordrand if i take them offsite they work, only onsite does this happen
03:24.28saint_yeah, so it s not the binary. definitely the network. try something crazy if you can. plug a phone directly to the NIC card on the PC. you might have to do a x-over cable and find a way to power the phone with a regular power brick instead of poe of course.
03:25.07saint_then add a network component between the server and the phone . if the trendnet causes the issue, then you know what to change.
03:25.09Bordrhaven't gone that far, placed a single 8 port / 4 port poe gig switch directly to the server and 2 phones
03:25.16Bordrthe problem persists
03:25.36saint_did you try to play with the sop option ?
03:25.41saint_s/sop/sip
03:25.42Bordrthe system has been working for 6 months and on Monday just started disconnecting
03:26.09Bordryou rtp/registration settings? not before the problem but have since.
03:26.15Bordrsorry you mean*
03:26.16saint_before you use sip option, do those phones support it ?
03:26.24Bordryes they do
03:26.28saint_no - sip option as "qualify=xxx" in sip.conf
03:26.42Bordryes qualify is yes and set to 120
03:26.45saint_try to add "qualify=yes" then
03:26.48Bordrtimeout of 2000
03:27.07saint_so try to remove it
03:27.11saint_and reload sip
03:27.44Bordrthat prevents it from showing in the logs, however, the pbx stumbles with the call and drops or goes silent for that time frame when the issue occurs
03:30.57saint_it does not make sense that a phone could not connect directly to the server with just one switch in between, but has no issue over WAN ..
03:31.10saint_could it be physical layer ?
03:31.22Bordrthat is why i'm stumped... over wan should be the problem not on net
03:31.27saint_seriously, try to connect a phone directly to the server. i would do that.
03:31.38saint_or i would use a known brand switch like alcatel-lucent
03:31.46Bordrdoubtful at that layer given all new hardware and wire runs
03:32.24Bordri pulled the managed switch out just for testing, they're phones are down now until i fix it
03:34.26Bordri will try the phone directly to the server tomorrow but don't see that changing anything. anywhere else I can look to get more info?
03:35.53saint_Sometimes weird things can happen. I used to manage a network with 120,000+ VoIP phones.. many times the issue was at the very basic level of things.
03:41.06Bordrif it were a layer problem though, why would I be able to continuously ping from the pbx to the phone even when the phone goes unreachable for 10 seconds?
03:43.11saint_some issues had know i had in the past were that some cisco would reset their ARP table, and while doing it, all connections would drop until it was ready to re-learn IP and MACs. I m not saying this is it, but by connecting directly to the server, you will see if it is because of the hardware in between. at leat.
03:44.33Bordrwill try like i said, this just feels like something wrong oserver side due to the wan users working fine and the fact i can ping while the problem is occuring
03:57.02*** join/#asterisk spicyramen (~Adium@c-50-184-3-185.hsd1.ca.comcast.net)
04:07.48cmendes0101I'm having some PDD issues with a vendor. Would there be any way I can setup logging specifically for this? Only other way I was thinking is to just log all SIP and just dig through
04:09.18youtmonhey guys GV module for freepbx is there one?
04:34.45*** join/#asterisk socomm (~socomm@71-92-196-23.static.mtpk.ca.charter.com)
04:36.08socommThis question gets asked here a lot Im sure, but can someone recommend me a good SIP trunk provider?
04:40.29[TK]D-FenderFlowroute , voip.ms, vitelity.net
04:41.32socomm[TK]D-Fender: Thank you.
04:52.52*** join/#asterisk luckman212 (~luckman21@unaffiliated/luckman212)
04:59.08*** part/#asterisk socomm (~socomm@71-92-196-23.static.mtpk.ca.charter.com)
05:15.46*** join/#asterisk MaliutaLap (nikolai@unaffiliated/maliuta)
05:32.03*** join/#asterisk mbecroft (~user@ak2.becroft.co.nz)
05:34.13*** join/#asterisk marlinc (~marlinc@unaffiliated/marlinc)
05:52.46*** join/#asterisk youtmon (~yout@c-73-46-212-142.hsd1.fl.comcast.net)
06:20.27*** join/#asterisk justdave (~dave@unaffiliated/justdave)
06:23.48*** join/#asterisk MaliutaLap_ (nikolai@unaffiliated/maliuta)
06:24.25*** join/#asterisk elitas (~elitas@213.226.135.203)
06:34.25*** join/#asterisk zapata (~zapata@2a02:b18:581:10:a9c4:21b8:7036:6a3)
06:37.14*** join/#asterisk bitjab (dodger@gateway/shell/devio.us/x-fngpevwrtsuufdvf)
06:47.03*** join/#asterisk tparcina (~tomo@212.92.200.41)
06:50.35*** join/#asterisk todwetsprock (~florian@p5099a594.dip0.t-ipconnect.de)
06:52.55*** join/#asterisk sparetire (~sparetire@unaffiliated/sparetire)
06:53.17*** join/#asterisk cryptic (~cryptic@142-196-132-50.res.bhn.net)
06:58.25*** join/#asterisk bof22 (~Thunderbi@185.13.183.107)
07:01.32*** join/#asterisk Marquel (~Marquel@fuchsfanclub/allerdings/marquel)
07:11.33*** join/#asterisk happy-dude (uid62780@gateway/web/irccloud.com/x-shoswljfkjvgaamn)
07:20.00Bordrsaint_ not sure if you are still around. have been digging through the errors and found a commonality... the phones go unreachable and the tcp dump indicates an ARP reqeust, in fact multiple arp requests. It looks like this is where the delay is coming from.
07:20.45Bordranyone out there run asterisk on centos and know how to diagnose an arp issue?
07:29.57*** join/#asterisk sommarnatt (~sommarnat@193.180.18.201)
07:38.35mbecroftBordr: If the phone is unreachable on its ethernet segment, then you will see multiple ARP requests as the server tries to locate the phone but gets no response
07:38.57mbecroftThis signifies a network fault or the phone has died/become disconnected from the network
07:40.15BordrI get that... The problem is I can ping the phone the entire time this happens. The phone will go unreachable for 2000+ms then reconnect 10 sec later. All phones exhibit this behavior if on local network. Wan users are not affected. Phones are Aastra 6867i and SPA 112 ATAs. Total of about 20 devices - 16 local 4 remote.
07:40.40wyoungBordr: fun
07:41.36BordrI have swapped switches, layed new wire, isolated down to a single 8 port swith with 4 port poe and nothing else (i.e. no wan, etc) and the problem persists
07:42.08Bordriperf gives 98% bandwidth and no issues in any logs :) lots of fun
07:42.11mbecroftBordr: are you pinging the phone from the same host running Asterisk?
07:42.16Bordryes
07:42.56mbecroftIt's not clear to me how you can be successfully pinging the phone while at the same time seeing repeated unanswered ARP requests--can you confirm that?
07:43.40mbecroftAnd paste the output of 'arp -n' for the phone in question during the fault condition
07:43.54Bordryes i can. and that is why i'm here :) I don't understand how that can be as well. Run continuous ping and max delay is 5ms yet asterisk times out at over 2000.
07:44.47BordrI have a TCPDUMP of that time period, bringing the server back up now. Moved it to another NIC and same issue so moving it back now.
07:44.49BordrARP, Request who-has 192.168.1.31 tell 192.168.1.35, length 46
07:44.49Bordr..........]H.....#...........
07:44.49Bordr................
07:44.49BordrARP, Reply 192.168.1.31 is-at 00:00:70:62:78:00, length 28
07:44.49Bordr..........pbx.......]H.....#
07:44.58BordrPBX is .31
07:45.14mbecroftOk so there was a reply.
07:45.35mbecroftAre you saying that gespite getting the ARP reply, the server was sending out repeated ARP requests?
07:45.38Bordryes, but it was too long acording to asterisk, however, grep of the full log shows:
07:45.40Bordr[2016-01-07 00:22:51] NOTICE[2662] chan_sip.c: Peer '117' is now UNREACHABLE!  Last qualify: 15
07:45.40Bordr[2016-01-07 00:23:01] NOTICE[2662] chan_sip.c: Peer '117' is now Reachable. (21ms / 2000ms)
07:45.41mbecroft*despite
07:45.58BordrYes it repeats several times 5-10.
07:46.25mbecroftTherea re no timestamps o nwhat you pasted; is the reply seen immediately after the request? And how long between requests?
07:46.26BordrLove the phone goes UNREACHABLE (which should mean 2000ms or more yet it was at 15).
07:46.46*** join/#asterisk tzafrir (~tzafrir@local.xorcom.com)
07:46.54BordrThe repsonses take about 2-3 seconds to complete all of them.
07:47.22mbecroftHave you disabled all iptables rules and set all tables to ACCEPT on the pbx?
07:47.43Bordrhaven't done that (KISS failed me again :) let me try
07:47.55mbecroftIt's odd that it would resend ARP requests after receiving a valid response
07:48.07mbecroftI take it you are running tcpdump on the pbx?
07:48.22BordrI agree. And yes on the PBX.
07:49.34mbecroftincidentally asterisk does not test reachability through ping as in ICMP echo request, but through a SIP protocol message
07:50.07mbecroftYou may have alreay answered this but, how often does the fault occur, and is it repeatable or intermittent?
07:50.34BordrThats news to me.  Was running full TCPDUMP on the PBX for all traffic not just sip when I saw the ARP repeat several times.
07:50.50BordrHere are the last few lines:
07:50.51Bordr[2016-01-07 00:08:04] NOTICE[2662] chan_sip.c: Peer '125' is now Lagged. (2052ms / 2000ms)
07:50.51Bordr[2016-01-07 00:08:14] NOTICE[2662] chan_sip.c: Peer '125' is now Reachable. (54ms / 2000ms)
07:50.51Bordr[2016-01-07 00:14:37] NOTICE[2662] chan_sip.c: Peer '117' is now UNREACHABLE!  Last qualify: 14
07:50.51Bordr[2016-01-07 00:14:47] NOTICE[2662] chan_sip.c: Peer '117' is now Reachable. (16ms / 2000ms)
07:50.51Bordr[2016-01-07 00:16:00] NOTICE[2662] chan_sip.c: Peer '111' is now UNREACHABLE!  Last qualify: 17
07:50.52Bordr[2016-01-07 00:16:10] NOTICE[2662] chan_sip.c: Peer '111' is now Reachable. (16ms / 2000ms)
07:50.54Bordr[2016-01-07 00:17:14] NOTICE[2662] chan_sip.c: Peer '111' is now UNREACHABLE!  Last qualify: 16
07:50.57Bordr[2016-01-07 00:17:24] NOTICE[2662] chan_sip.c: Peer '111' is now Reachable. (16ms / 2000ms)
07:50.59Bordr[2016-01-07 00:22:51] NOTICE[2662] chan_sip.c: Peer '117' is now UNREACHABLE!  Last qualify: 15
07:51.01Bordr[2016-01-07 00:23:01] NOTICE[2662] chan_sip.c: Peer '117' is now Reachable. (21ms / 2000ms)
07:51.41BordrIntermittent, or more like random. Have no way to reproduce it and sometimes it goes 15 minutes without showing up other times 3-5 seconds and another goes offlien.
07:51.56mbecroftSo it'll almost certainly happen within 15 minutes?
07:52.51mbecroftDoes this happen with more than one make and model of phone?
07:53.15*** join/#asterisk luckman212 (~luckman21@unaffiliated/luckman212)
07:53.15BordrYes with Aastra 6867i and Cisco SPA 112
07:53.35BordrPBX has been in prod for 6+ months and just started dropping calls on Monday evening.
07:53.53mbecroftAh, so it's been fine for months and just started playing up
07:54.07mbecroftAnd you say you tried another NIC in the server?
07:54.10Bordryes. problem is I can't find anything that has changed.
07:54.17Bordryes and didn't help
07:54.30mbecroftNot just a different port on the same NIC, but a separate card?
07:54.43Bordrcorrect, from broadcom to intel
07:54.57mbecroftAt this point, depending on the urgency, I would just be swapping out the server hardware
07:55.13*** join/#asterisk pawiecki (~pawiecki@217.97.180.1)
07:55.32BordrThat is what I have scheduled for tomorrow morning. Don't really want to go there if I can help it (lots of rebuild time needed).
07:55.48mbecroftHopefully you could just dd the disk image across or whatever
07:56.17mbecroftThe other thing I suggest is doing a full tcpdump capture and analysing it in wireshark
07:56.19BordrI thought about that, but it seems more likely a problem in the server os than hardware.
07:56.39mbecroftI'm having trouble imagining an OS issue that will cause this
07:56.58mbecroftSpecificaly, analyse the ARP reponses in wireshark to see if they are valid and correct
07:58.04BordrSo am I. That's why i'm here :) Am able to blast the network at near full GB without any issues and can push for 4 hours straight without issue on iperf. If it was hardware I would think it wouldn't hold up to that.
07:58.24mbecroftBecause if you are seeing a valid and correct arp response, it is not being dropped by iptables or rp_filter, arriving at the server, there is no reason it should then retry the arp request
07:58.34mbecroftIf the arp response is corrupt, that would explain things
08:00.25BordrI agree. If the client is receiving the ARP request normally it wouldn't request it agian.
08:00.48BordrJust dropped iptables and started a filter for the peer status... Should show in a few minutes.
08:00.57*** join/#asterisk pchero_work (~pchero@109.70.54.56)
08:01.54Bordr[2016-01-07 01:01:42] NOTICE[2644] chan_sip.c: Peer '111' is now UNREACHABLE!  Last qualify: 15
08:01.54Bordr[2016-01-07 01:01:43] NOTICE[2644] chan_sip.c: Peer '125' is now UNREACHABLE!  Last qualify: 60
08:02.02Bordrdidn't get the tcp dump going yet though.
08:02.35mbecroftuse wireshark, or use tcpdump to capture to a file you can later analyse in wireshark
08:03.19Bordrdidn't know you could analyse tcpdump in wireshark
08:03.22*** join/#asterisk lumasepa (~gestoip@193.145.124.30.local.ull.es)
08:03.27mbecroftSo at present the server is completely isolated from the core network, and attached only to a switch with the phone plugged into it?
08:03.50mbecroftTO do that you need to do tcpdump -w filename
08:04.10mbecroftTo be safe, depending on the version, you may need to also add -s 0
08:04.11Bordrpretty much, i have access to it via DRAC but thats it, no internet 3 phones and 1 ATA plugged in now
08:04.37mbecroftYou definitely don't have any IP address conflicts between those devices?
08:05.01Bordrnope... pbx on .31 and phones on .35, .36, .37, ata at .32
08:05.38mbecroftAre the multiple arp requests time correlate with the fault seen at the asterisk level?
08:06.08Bordryes, I just found that when I posted here. The last 3 errors all occured at the arp request.
08:06.22mbecroftThat seems suspicious
08:06.35mbecroftI would like to see a wireshark analysis of the ARP responses
08:06.43BordrHere is hte ARP and it is still going....
08:06.44BordrARP, Request who-has 192.168.1.35 tell 192.168.1.31, length 28
08:06.44Bordr...........,.l.............#
08:06.44BordrARP, Request who-has 192.168.1.31 tell 192.168.1.35, length 46
08:06.44Bordr..........]H.....#..........J.................
08:06.44BordrARP, Reply 192.168.1.31 is-at fe:a9:07:2c:17:6c, length 28
08:06.46Bordr...........,.l......]H.....#
08:06.48BordrARP, Request who-has 192.168.1.31 tell 192.168.1.35, length 46
08:06.49mbecroftBut I'm finding it hard to imagine what fault could cause this
08:06.51Bordr..........]H.....#..........J.................
08:06.53BordrARP, Reply 192.168.1.31 is-at fe:a9:07:2c:17:6c, length 28
08:06.55Bordr...........,.l......]H.....#
08:07.04mbecroftNone of that is any help, we need to analyse the full packet in wireshark
08:07.05Bordrtrying to get that now
08:07.49mbecroftHow long is it down before it comes right?
08:08.26Bordrpanel shows 10 seconds but phones start playing audio correctly (if call hasn't terminated) within about 6 seconds
08:08.48Bordrcapturing now, and waiting for another error
08:09.24mbecroftHave ping running as well, to the same phone that exhibits the fault
08:09.39mbecroftI guess you don't know in advance which one is going to play up next
08:09.51mbecroftThen let's analyse the full packet capure
08:10.33BordrIt seems to pick on extension 117 more than the others (right now anyway)... About 4 hours ago it was 111 that it hated.
08:12.28mbecroftIn production, are phones on the same network segment as the pbx, or is there one or more routers inbetween?
08:13.03Bordrflat network, nothing between but 2 switches (they have been taken out now and a simple non-managed 8port put in for diag)
08:14.12*** join/#asterisk hehol (~hehol@gatekeeper.loca.net)
08:14.30Bordrext 125 just went unreachable... waiting for it to come back up
08:14.41Bordrit's back up
08:21.56*** join/#asterisk Chainsaw (~chainsaw@gentoo/developer/chainsaw)
08:25.12Bordrmbecroft: are you still there?
08:25.22Bordrthink i found something...
08:27.15Bordr<PROTECTED>
08:27.15Bordr<PROTECTED>
08:27.15Bordr<PROTECTED>
08:27.15Bordr<PROTECTED>
08:27.15BordrAddress Resolution Protocol (reply)
08:27.16Bordr<PROTECTED>
08:27.18Bordr<PROTECTED>
08:27.20Bordr<PROTECTED>
08:27.20mbecrofthi
08:27.22Bordr<PROTECTED>
08:27.24Bordr<PROTECTED>
08:27.27Bordr<PROTECTED>
08:27.29Bordr<PROTECTED>
08:27.31Bordr<PROTECTED>
08:27.33Bordr<PROTECTED>
08:27.59mbecroftHave you upgraded the pbx OS recently?
08:28.14Bordrthis evening it did
08:28.26Bordrotherwise not in a while (months)
08:28.49mbecroftI was looking for bad checksum on the packet
08:28.49Bordrthat error above occured 2 seconds before it posted unavailable
08:29.13mbecroftAre you seeing that on all packets are just some?
08:29.19mbecroft*or
08:29.42Bordronly that one at 2.01 seconds before disconnect
08:30.22Bordrand appearantly i brought up the nic online... lots more here than i thought.
08:30.42Bordrthe only other is 3164330.84507473.14.237.142192.168.1.31ICMP70Destination unreachable (Communication administratively filtered)
08:30.48mbecroftThe 2nd NIC you tried replacing with, is it from the same or a different vendor than the 1st NIC?
08:31.03Bordrdifferent. primary was broadcom and secondary was intel
08:32.07mbecroftThat kind of eliminates this, but for the sake of it, try disabling all offload on the NIC
08:32.23Bordrit's already off (that one i did think of)
08:32.29BordrI just found this in here too:
08:32.31Bordr3366353.513011CiscoInc_32:8f:c6fe:a9:07:2c:17:6cARP64Who has 192.168.1.31? Tell 192.168.1.32 [ETHERNET FRAME CHECK SEQUENCE INCORRECT]
08:32.41Bordrand then just after:
08:32.42Bordr3367353.513050fe:a9:07:2c:17:6cCiscoInc_32:8f:c6ARP42192.168.1.31 is at fe:a9:07:2c:17:6c
08:33.03Bordrthe first one has all 0's for the mac, the 2nd filled in the mac
08:33.07mbecroftSo it's just the occasional frame with invalid FCS
08:33.54mbecroftThis doesn't make sense, because you've replaced all the hardware that could result in link level data corruption and you've both disabled offload and tried a different NIC
08:34.23mbecroftJust to be clear, this problem happens with any phone, not just the ones you happen to be testing with right?
08:35.03mbecroftI mean the one thing we haven't done is replace the phone, but it's hard to imagine how a whole stack of phones would suddenly develop networking faults at the same time
08:35.34mbecroftReplace the cables
08:36.06mbecroftBut the kind of corruption we are seeing (FCS = 0 etc.) suggests a software/firmware/switching silicon issue
08:37.58Bordragreed. wierdest part is if I take the phones offsite they work fine. The one on my desk hasn't been lagged yet (exept changing the nic). Swapped it today for another phone. They all work fine over WAN it's only the LAN - thats why I replaced the cables and switching figuring it had to be one or the other.
08:39.33mbecroftWell this is the thing--if you put a router between the pbx and the phones they will work fine--because clearly this is a L2 issue
08:40.25BordrThat's my point, not sure what is left to change. New switch, new cables - nothing else between the 2 devices.
08:40.54mbecroftThere is some kind of problem at layer two when these particular phones are used with this particular pbx hardware
08:41.30mbecroftHow easy is it for you to change routing and reprovision phones? Is that an option?
08:41.43BordrI'll image the PBX to a new box in the morning to see if it resolves it. If not build from scratch to see.
08:41.51BordrYeah, that's easy enough takes about 2 minutes
08:42.01mbecroftThen one potential solution is:
08:42.10mbecroftput the pbx onto a differen subnet from the phones
08:42.25mbecroft(hence it will have a new IP address, so the phones will need to be reprovisioned)
08:42.45mbecroftassuming you already have a router, configure it to route between the PBX network and the phone network
08:42.57Bordrgood idea, have them register via the public IP over NAT...
08:43.01mbecroftIdeally they would be on separate VLANs but the two networks can even be on the same network segment
08:43.29mbecroftThis will almost certainly get things working, since the problem seems happen only when the phones are on the same network segment as the pbx
08:43.36mbecroftI bet if there is a router in between, they will work
08:44.13mbecroftHow many phones in total?
08:44.35Bordr20
08:44.42mbecroftHaving them register to the public IP with NAT might be the quick fix you need, but may fall down if there are hundreds of phones
08:44.48mbecroftOh, in that case just do it
08:45.01Bordrmaking the changes now
08:45.04mbecroftReprovision all the phones to connect via the public IP address. See if the problem goes away
08:45.35mbecroftI'm assuming that this will mean they go via your router
08:46.17Bordrshould. it should be set for hairpin natting
08:46.20mbecroftIt leaves the underlying problem unresolved and I still suggest moving the pbx to new hardware when possible
08:46.43Bordryes, but i won't have to do it during business hours... i can backup and test on the bench
08:46.56mbecroftIndeed, if the quick fix works!
08:47.48Bordrare you in the US?
08:49.07*** join/#asterisk youtmon (~yout@c-73-46-212-142.hsd1.fl.comcast.net)
08:50.50hagbardmbecroft: FWIW, with some operating systems and some NICs, the frame check sequence is verified in the ethernet hardware and, due to driver/hardware details, is not preserved when the packet is handed over to the OS. It's relatively common.
08:51.13mbecrofthagbard: True, but here we are seeing only some packets with 0 FCS
08:51.22hagbardmbecroft: Are they only in a specific direction?
08:52.00mbecroftFrom what I understand it's somewhat at random, there are just occasional packes that will have invalid FCS. Haven't verified if all in same direction--good question.
08:52.08Bordryes they appear to be, only have the 2 instances to review but the tcpdump indicates an all 0's mac for the devices
08:52.17hagbardNICs that do TX checksum offloading, the OS wont bother computing a checksum - for those, a 0 FCS is expected.
08:52.30mbecrofthagbard: We have disabled all offloading
08:53.02hagbardUnderstood.
08:53.04mbecroftThe all 0's mac is another odd one
08:53.21hagbardAn all 0 MAC, on the otherhand, is quite strange.
08:53.44mbecroftI sense a problem in the network interfaces or switching silicon. But all of those have been swapped out
08:54.40hagbardIncidentally, the fe:a9:07:2c:17:6c MAC address from above is also equally strange - it's a "locally administered" address.
08:55.17hagbardIf its a managed switch, I'd check the interface error counters.
08:55.24mbecroftIs 192.168.1.31 the pbx host?
08:55.47mbecroftCurrently it's been moved off the managed switch onto an unmanaged switch, to eliminate the switch as a possible cause
08:55.50Bordrit was but it was pulled out to diag.... there were 2 errors on the link to the pbx in the last 30 days
08:55.57Bordryes .31 is the pbx
08:56.00mbecroftI agree with putting it back onto the managed switch and then monitoring the error count
08:57.04mbecroft<PROTECTED>
08:57.49Bordr[root@west-pbx tmp]# /sbin/ip addr list eth0
08:57.49Bordr2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
08:57.49Bordr<PROTECTED>
08:57.49Bordr<PROTECTED>
08:57.49Bordr<PROTECTED>
08:57.51Bordr<PROTECTED>
08:58.06hagbardethtool -i eth0
08:58.09hagbardif you have ethtool
08:58.45Bordr[root@west-pbx tmp]# ethtool -i eth0
08:58.45Bordrdriver: 8139cp
08:58.45Bordrversion: 1.3
08:58.45Bordrfirmware-version:
08:58.45Bordrbus-info: 0000:00:12.0
08:58.46Bordrsupports-statistics: yes
08:58.48Bordrsupports-test: no
08:58.50Bordrsupports-eeprom-access: yes
08:58.52Bordrsupports-register-dump: yes
08:58.55Bordrsupports-priv-flags: no
08:58.58hagbardIs this a VM?
08:59.02Bordrnormally I would use broadcom or intel, this card was left over for testing
08:59.03mbecroftI agree the MAC is odd
08:59.26hagbardBordr: I have not seen a realtek 8139 in a decade.
08:59.38mbecroftI assumed it wasn't a VM since he was talking about swapping NICs etc
09:00.05hagbardUnderstood, but a real nic with a generated MAC?
09:00.16mbecroftYes, this is very strange
09:00.24Bordryes its a vm, nic is dedicated.
09:00.31mbecroftAh
09:00.38hagbardI do believe virtualbox/kvm virtualization offers an 8139 as a "virtual nic prototype."
09:00.50mbecroftOk, that explains a few things
09:01.01hagbardBordr: In the future, if available, I'd recommend you use the e1000 "virtual ethernet" adapter.
09:01.14Bordryes, they do, I only map appropriate drivers to the machine using the nic.
09:01.23mbecroftIf you are using kvm, I woul recommend using virtio networking
09:01.26mbecroftbut anyway
09:01.27BordrWe have always used e1000 or virtio.
09:01.32hagbardEven though the hardware is virtualized, the e1000 has a better programming interface - there's less overhead.
09:01.50hagbardHere you appear to be using realtek 8139, an ancient 10/100 nic.
09:01.57BordrFor fast small connection e1000 works best, virtio for heavy io
09:02.14Bordryes it's mapped to a rtl nic in the server
09:02.29hagbardOh, as PCI passthrough?
09:02.36mbecroftWhen you say that, do you mean the NIC is PCI passed through to the VM?
09:03.03BordrWe passthrough the broadcoms and the intels, this one i think I just mapped (was getting frustrated onsite).
09:03.16mbecroft"mapped" meaning?
09:03.47BordrWe assign the nic to vmbrx and the only machine using that interface is the vm.
09:03.54mbecroftI see
09:04.04mbecroftIt certainly appears you are *not* using PCI passthrough for the NIC at present
09:04.06hagbardBut the physical nic coincidentally is also an 8139?
09:04.10BordrCorrect.
09:04.24BordrNo it's a rtl but not an 8139.
09:04.34hagbardIn that case, I'd still recommend using the e1000 "virtual hardware" and mappping it to the 8139 on the host.
09:04.54BordrI can change that - however, mapped and passthrough exhibit the same issue.
09:04.57mbecroftBut the fault started happening prior, when you had an actual NIC with PCI pass-through to the vm?
09:05.05BordrYes
09:05.40mbecroftNormally at this point I would blame bugs in the VM stack, but the fact it also happens on the passed-through adapter seems to rule that out
09:05.56BordrTo be honest it was happnening so fast my first thought was a switching loop. Got onsite expecting someone had screwed with something but all was well wiring wise.
09:06.04hagbardpass-through NICs aren't above reproach. There's still quite a lot of hand holding.
09:06.31mbecroftBut one would not expect exactly the same problem in both cases
09:06.58mbecroftI also suggest running memtest86 on the host overnight if possible
09:07.09hagbardBordr: Are you familiar with the arping utility?
09:07.24BordrAgreed hagbard. I have tried mapping individual nics, and passthrough on both broadcom, intel and only mapping on the rtl.
09:07.52BordrI'm sure I am but you may need to refresh me a bit... It sounds familiar
09:07.59mbecroftBordr: I think I will hand you over to hagbard as it's getting late here :)
09:08.18hagbardIt's 4am here, I'm out in a few. But, this is an interesting problem.
09:08.21BordrTY for the help mbecroft! They are registered now and no errors yet.
09:08.34BordrYou must be east coast... 2:00am here :)
09:08.35hagbardBordr: So, arping is a similar idea to ping, but it works by sending an ARP request.
09:08.59hagbardBordr: Your original problem statement described ARP requests suddenly timing out for a period of time
09:09.07Bordryeah i remember now, we used to use it a few years ago.
09:09.07mbecroftPlease report back to me if you eventually figure out what the problem is!
09:09.14Bordrwill do
09:09.28hagbardAre you able to yum install arping if it's not available?
09:09.34Bordrit's already there
09:09.56hagbardCould you start an arping for an example problematic phone and leave it running for a few minutes?
09:10.09hagbardDepending on the version, you may need to specify your nic
09:10.10hagbardie
09:10.16hagbardarping -I eth0 <phone IP>
09:10.46Bordrit's going now
09:11.02Bordrran 3 then long pause
09:11.06hagbardYou have access to the VM host as well - is it a linux/kvm machine or xen or something?
09:11.12Bordrkvm
09:11.16Bordryes
09:11.31hagbardCan you check for any error counters on the host interface
09:11.46hagbardjust ifconfig eth0
09:11.51hagbardoverruns etc.
09:11.57*** join/#asterisk Jesterboxboy (~Thunderbi@chello080109194026.3.graz.surfer.at)
09:12.02Bordr<PROTECTED>
09:12.02Bordr<PROTECTED>
09:13.07hagbardok, the 8139 may have extended error counters
09:13.09hagbardethtool -S eth0
09:13.24hagbardAre any of those non zero? (No need to cut and paste.)
09:13.51BordrNIC statistics:
09:13.51Bordr<PROTECTED>
09:13.51Bordr<PROTECTED>
09:13.51Bordr<PROTECTED>
09:13.51Bordr<PROTECTED>
09:13.51Bordr<PROTECTED>
09:13.54Bordr<PROTECTED>
09:13.56Bordr<PROTECTED>
09:13.58Bordr<PROTECTED>
09:14.00Bordr<PROTECTED>
09:14.02Bordr<PROTECTED>
09:14.05Bordr<PROTECTED>
09:14.07Bordr<PROTECTED>
09:14.09Bordr<PROTECTED>
09:14.11Bordr<PROTECTED>
09:14.16Bordrrx_err is high
09:14.50hagbardIt looks that way.
09:15.07hagbardwhile sleep 30; do ethtool -S eth0 | grep rx_err ; done
09:15.16hagbardThat should show you the value of rx_err every 30 seconds
09:15.30hagbardif that increments steadily, then I think you should find a different nic.
09:15.48hagbardI do appreciate that your problem predates the use of this 8139.
09:15.59hagbardSo, I suspect that you have an underlying network issue.
09:16.58BordrThu Jan  7 02:16:30 MST 2016
09:16.58Bordr<PROTECTED>
09:16.58BordrThu Jan  7 02:16:45 MST 2016
09:16.58Bordr<PROTECTED>
09:17.01hagbardThe arping you left running - you said the phone stopped responding, has it resumed responding? If so, how long was it quiet?
09:17.25Bordrnot the phone, the arping itself just hung then resumed
09:18.10hagbardSome arping versions will print out if there's a timeout, others won't print out anything if there's no response.
09:18.27hagbardThe important thing is the value for index= before and after the "hang"
09:18.36Bordrthats what it seemed like, it was responding a rate then stopped for about 2 then resumed.
09:18.49Bordrit shows no index
09:18.55BordrUnicast reply from 192.168.1.32 [B0:FA:EB:32:8F:C6]  1.349ms
09:18.55BordrUnicast reply from 192.168.1.32 [B0:FA:EB:32:8F:C6]  1.321ms
09:19.26hagbardJussec, let me check if there's a flag you can use.
09:20.36Bordr-u shows index
09:20.58Bordrbut not on my machine i guess :)
09:21.10hagbardDifferent versions of the utility witht he same name.
09:21.21hagbardIn debian, there's an arping package and an iputils-arping
09:21.40hagbardThe one you're using would be the one called iputils-arping in Debian
09:21.41Bordron centos here
09:21.48hagbardI appreciate that.
09:21.59hagbardcan you yum search arping and see if they have an alternate?
09:23.18hagbardOh! Interesting.
09:23.42Bordrnot finding anything. Yes sir ?
09:23.50hagbardIs there an "arping" package.
09:24.01hagbardYour arping binary is probably part of the iputils package.
09:24.03hagbard60 bytes from 28:c6:8e:c0:e2:21 (192.168.2.1): index=2 time=3.658 msec
09:24.13hagbardThats what the output looks like from my version.
09:24.53hagbardIf arping stops receiving ARP replies consistently for the same period of time - like it always cuts out for 10 seconds or 45 seconds at a time, that's a very worthwhile clue.
09:25.45BordrWell i put a 3 second timer on it...
09:25.45Bordr[root@west-pbx tmp]# arping -w 2 192.168.1.32
09:25.45BordrARPING 192.168.1.32 from 192.168.1.31 eth0
09:25.45BordrSent 3 probes (3 broadcast(s))
09:25.45BordrReceived 0 response(s)
09:25.54Bordrsorry 2 second...
09:26.01hagbardJust let it run.
09:26.09hagbardwithout a wait.
09:26.27BordrI don't have a counter then :)
09:27.04hagbardAgain, if you can get the alternate arping version (I can build you a static binary if you want.) it gives you a count, like regular ping does.
09:28.21hagbardOh, an alternative idea.
09:28.22hagbarddo
09:28.40hagbardarp -s 192.168.1.32 B0:FA:EB:32:8F:C6
09:28.45hagbardand then start a regular ping
09:29.19hagbardarp -s manually configures the MAC address, so no more problems with ARP.
09:29.26Bordrnot getting anything
09:29.34Bordrit skipped the first 15 pings
09:30.08hagbardJust let it run
09:30.11hagbardfor like 5 minutes.
09:30.30hagbard45 seconds of silence, suggests a spanning tree issue.
09:30.56hagbard15 seconds of silence suggests a different kind of spanning tree issue.
09:30.57Bordrjust paused for over 30 but not 45
09:32.50hagbardWell, I suspect a layer 2 issue.
09:32.56hagbardEspecially if its intermittent in nature.
09:33.26Bordrright there with ya. paused again 15 seconds
09:33.41hagbardcrap, 4:30. I gotta sleep.
09:33.54hagbardWhat about a different phone?
09:33.59Bordrsounds good, thanks for the help. will replace the server tomorrow and see where I get.
09:34.01hagbardjust to be thorough
09:34.09Bordrdoes it to all 20 but only when they are onsite
09:34.15Bordroffsite they work fine
09:34.21hagbardok.
09:34.27Bordralso does the same on the ATAs
09:35.16hagbardBtw, when you pull that 8139 out of the machine, throw it away. It really is a terrible NIC. Intel eepro 1000 NICs are so cheap that it's a crime to use that realtek.
09:35.47hagbardEspecially if you're using a DRAC, that means you should have at least 2 gigE onboard and, if its a newer poweredge, four.
09:35.48Bordrlol, there is an e1000. i was throwing everything i could find at it to troubleshoot
09:36.14Bordr64 bytes from 192.168.1.32: icmp_seq=363 ttl=64 time=1.26 ms
09:36.15Bordr64 bytes from 192.168.1.32: icmp_seq=364 ttl=64 time=1094 ms
09:36.15Bordr64 bytes from 192.168.1.32: icmp_seq=365 ttl=64 time=96.0 ms
09:36.15Bordr64 bytes from 192.168.1.32: icmp_seq=366 ttl=64 time=1.33 ms
09:36.20Bordrpings are going crazy
09:37.22hagbardAny other machines on that network? If they show the same intermittent behavior with arping and that phone, I'd suspect the switches
09:37.58Bordrnot that are on. they are all sleeping and are wifi so magic packets
09:38.16hagbardwait, wifi
09:38.23hagbardI'd assumed this was all ethernet.
09:38.51Bordrit is, the only other computers that are there are on wifi - all other ethernet devices were unplugged
09:39.01hagbardAhh, understood. My mistake.
09:39.10Bordrphones and ata don't use wifi :) that would be really bad...
09:40.32hagbardwifi can vary dramatically between reliable and terrible - depends a lot on the noise floor, amount of interference, number of nearby shitty microwaves, etc.
09:41.01hagbardIncidentally, the rx_err on the host 8139 would make me literally throw the card away.
09:41.23Bordralright it'll hit the trash tomorrow morning...
09:41.37hagbardGood night, good luck.
09:41.46hagbardIf you figure it out, I'd appreciate /msg with the details.
09:41.53Bordrwill do
10:42.00*** join/#asterisk luckman212 (~luckman21@unaffiliated/luckman212)
10:52.56snadgei wonder why zoiper makes asterisk 11 do this: ast_rtp_read: Unknown RTP codec 95 received from
10:53.09snadgei cant find any reference to codec 95.. its in an unassigned range
10:53.22snadgeit still works anyway.. just curious
11:13.05Tim_Toadysnadge: probably some video codec (jusging by the number) in the SDP that has no a=rtpmap attribute
11:13.30Tim_Toadyand asterisk cannot map it to some codec on its part
11:15.07*** join/#asterisk marceloamorim (~marcelo@189-90-192-72.isimples.com.br)
11:22.03snadgezoiper does video? hmm
11:26.13Tim_Toadymaybe not, its just a guess, high codes are usualy video codecs, but to be sure you need a trace with the INVITE that contains the SDP info
11:36.08*** join/#asterisk bof22 (~Thunderbi@185.13.183.107)
11:50.49*** join/#asterisk youtmon (~yout@c-73-46-212-142.hsd1.fl.comcast.net)
11:54.56DanQuinneyhey guys, just a (hopefully) quick question, we're setting `Monitor` on incoming calls with the options m and b: Monitor(wav,${CDR(recording_path)},mb) however if we use the playback command to play a file to callers the call isn't recording - am I missing something or can I force it to record these calls?
11:56.34WIMPyYes, you told it not to.
11:56.48WIMPyWhen there's Playback, that's not a bridged call.
11:58.42DanQuinney:)
11:59.11DanQuinneyyes I told it so it doesn't record unanswered calls - can I not make it do both? ;)
11:59.39WIMPyStart recording on asnwer.
12:06.34DanQuinneycool, cheers :)
12:15.12*** join/#asterisk kayatwork (~kayfox@orca.zerda.net)
12:23.34*** join/#asterisk kayatwork (~kayfox@orca.zerda.net)
12:23.40*** part/#asterisk marceloamorim (~marcelo@189-90-192-72.isimples.com.br)
12:36.33*** join/#asterisk vinrock (~vin@unaffiliated/vinrock)
12:36.44*** join/#asterisk kayatwork (~kayfox@orca.zerda.net)
12:45.45*** join/#asterisk marceloamorim (~marcelo@189-90-194-35.isimples.com.br)
12:54.56*** join/#asterisk italorossi (~Adium@177.65.201.177)
12:57.05*** join/#asterisk aurs (~aurs@84.48.191.51)
12:59.21*** join/#asterisk fstd (~fstd@unaffiliated/fisted)
13:05.39*** part/#asterisk marceloamorim (~marcelo@189-90-194-35.isimples.com.br)
13:28.51*** join/#asterisk Oatmeal (~Suzeanne@75-103-145-152.ccrtc.com)
13:33.24*** join/#asterisk davlefouAMD (~davlefouA@41.225.212.24)
13:43.53*** join/#asterisk [TK]D-Fender (~chatzilla@216-191-106-163.dedicated.allstream.net)
13:44.20*** join/#asterisk Draecos (~Draecos@203-121-194-11.e-wire.net.au)
13:46.06*** join/#asterisk marceloamorim (~marcelo@189-90-194-35.isimples.com.br)
13:56.33*** join/#asterisk bof22 (~Thunderbi@185.13.183.107)
13:58.54*** join/#asterisk mcargile (~mikec@rrcs-97-76-33-146.se.biz.rr.com)
14:02.17*** join/#asterisk simplydrew (~simplydre@unaffiliated/simplydrew)
14:03.27*** join/#asterisk sommarnatt (~sommarnat@193.180.18.253)
14:05.28*** join/#asterisk CeBe (~CeBe@a81-14-224-229.net-htp.de)
14:07.04*** join/#asterisk simplydrew_ (~simplydre@unaffiliated/simplydrew)
14:08.22*** join/#asterisk bof22 (~Thunderbi@185.13.183.107)
14:08.38*** join/#asterisk jasonwert (~wert@75-134-81-98.static.aldl.mi.charter.com)
14:10.21*** join/#asterisk brad_mssw (~brad@66.129.88.50)
14:12.37*** join/#asterisk bof22 (~Thunderbi@185.13.183.107)
14:14.04*** join/#asterisk bof22 (~Thunderbi@185.13.183.107)
14:26.22*** join/#asterisk happy-dude (uid62780@gateway/web/irccloud.com/x-jggiocwbushgcape)
14:32.47zambahi guys.. previously it was the case that the trunks set up between asterisk and lync only could use the ulaw/alaw codec.. has there been any development in this area? i know this is a bit of an open question, but i was hoping to lure out the people in here that have integrated asterisk with lync and may have some experiences
14:33.34WIMPyAsterisk supports a lot of CODECs, don't know about lync.
14:34.17[TK]D-Fenderhttps://technet.microsoft.com/en-ca/library/jj688118%28v=ocs.15%29.aspx
14:35.13[TK]D-FenderI see more than just G.711 there
14:35.56WIMPyYes, but not that much.
14:38.14[TK]D-FenderAgreed.
14:38.51WIMPyBut at least you can get G.722. Not that bad.
14:39.08zambawhich one of these do you suggest for the lync-asterisk integration?
14:39.46WIMPyWhat do your phones support?
14:39.47[TK]D-FenderWhatever is closest to what your other devices use
14:40.08[TK]D-FenderBecause will have to transcode if they don't match and someone is going to be losing out
14:40.18[TK]D-Fender#raresense
14:40.20zambaour polycom phones support g.722, i think
14:40.25zambathat's "HD Voice", right?
14:40.42WIMPyYes. It has many names.
14:46.25*** join/#asterisk Naikrovek (~jjohnson@204.54.36.245)
14:48.11zambaand that's supported by asterisk?
14:48.27WIMPysure
14:49.00zambain the 11.x train as well?
14:49.19*** join/#asterisk Jesterboxboy (~Thunderbi@chello080109194026.3.graz.surfer.at)
14:49.27WIMPyA lot longer.
14:49.34WIMPyMaybe forever.
14:49.39zambawhat's the asterisk name of that codec?
14:49.54WIMPyTake a guess.
14:50.29zambaG.722 = g722 (don't confuse this with g722.1 or g722.2)
14:50.48zambag722, i guess.. why the comment about confusing the codecs?
14:51.25WIMPyBecause G.722.1 and G.722.2 are not related to G.722 other than the similar name.
14:52.52*** join/#asterisk youtmon (~yout@c-73-46-212-142.hsd1.fl.comcast.net)
14:53.25zambahow is the upgrade from 11.x to 12.x?
14:53.32[TK]D-Fenderfine
14:53.37[TK]D-FenderExcept you shouldn't
14:53.41zamba[TK]D-Fender: oh?
14:53.42[TK]D-FenderBecause 12 is DEAD
14:53.54zambaah, go for 13 instead?
14:53.59[TK]D-FenderClearly
14:54.06zambacan i go directly from 11.x to 13.x?
14:54.10file12 was a standard release with a limited lifetime, which helped to shape up 13
14:54.13Naikrovekyeah duh *snort* /sarcasm
14:54.16WIMPycould easily argue that Asterisk is dead, but that wouldn't help.
14:54.38Naikrovekhasn't paid attention to asterisk in years.
14:55.29zambaWIMPy: oh, why do you say that?
14:56.37WIMPyThere used to be lots of 3rd party add-ons. Almost all of which have gone together with their developers.
14:58.17filefrom an Asterisk perspective since our move to git and gerrit we've been seeing more people submit patches and contribute, a lot of new people
14:59.01WIMPyStill all the good old stuff is gone.
15:01.15Naikrovekfile, that is good to hear.
15:01.17*** join/#asterisk u0m3 (~u0m3@188.27.154.248)
15:01.46fileI'll also say that people seem to be writing applications using the new ARI functionality for themselves or companies that have hired them, but are not necessarily making that available as a paid application
15:04.38fileanyway, just some comments based on what I've seen or heard
15:05.37WIMPyI hear people leaving because of the instability and I understand that very well.
15:06.40*** join/#asterisk kharwell (kharwell@nat/digium/x-wjbssorcxzbpsgdb)
15:07.00fileultimately it depends on what you use in Asterisk as to how stable it is, we try to make sure that everything core and even a bit outside of that is as stable as it can be but that doesn't cover it all
15:08.10WIMPyIt's the internals that are unstable. Constantly becoming incompatible to previous versions.
15:08.44filedo you mean that the APIs are unstable in their interface?
15:08.56WIMPyYes
15:09.08Naikrovekthat's not the "unstable" I thought you meant.  that's more "consistency"
15:09.12fileyes, they change between major versions
15:09.23filebut that's allowed us to implement functionality people have wanted for years and to make it better over all
15:09.25WIMPyThat seems to be the reason for the 3rd parties to have left. Some have clearly said so.
15:09.47Naikrovekthat's what major version numbers are for, yeah?  that's where big changes are supposed to go, in major revision changes
15:09.59WIMPyWould have been better to make a clear cut instead of doing it over and over again.
15:10.09filecan't foresee the future
15:10.32WIMPyIt's history.
15:10.33filewhile API changes may still occur in the future, they probably won't be as large as 11->12
15:11.07Naikrovekas a software developer I understand file's perspective.  As a customer I understand WIMPy.  my brain is fighting with itself.  now i have a headache.
15:11.08WIMPyPrevious ones seemed to have be a lot worse.
15:11.40filewe try to maintain backwards where we can, but it's impossible or a TON of work is required to do so
15:11.42WIMPyActually I'm arguing from the developers perspective.
15:12.20WIMPyAs A user I just see new features coming and old ones go.
15:12.52*** join/#asterisk Oatmeal (~Suzeanne@75-103-145-152.ccrtc.com)
15:12.54filenew features in Asterisk, or the add-ons you are referring to?
15:13.18WIMPyIn Asterisk.
15:13.34filewhat's some examples?
15:13.42WIMPyThe 3rd party stuff is usually just abandoned.
15:14.03WIMPyThe 1st big one was the removal of the ENUM switch.
15:14.27WIMPyOff course switches don't really work anyway, but that was a bit of a bummer.
15:14.42fileuh did that happen years ago?
15:15.03WIMPyAnd my concern at the moment is that DTMF detection is broken with native bridges in 11.
15:15.10WIMPyYes, it was long ago.
15:15.33filewe don't remove stuff anymore, unless absolutely necessary
15:16.10fileand do you have the ASTERISK issue for the native bridging?
15:16.58WIMPyNo. It looks like it was intentional. chan_dahdi disables native bridges when DTMF is required with a comment saying that it's broken.
15:17.10fileokay, so it's DAHDI
15:17.23filewell - DAHDI is involved
15:17.30[TK]D-FenderDAHDI issues .... classic
15:17.56WIMPyNo, I use them with chan_lcr. But chan_dahdi was where I found out that it's not about chan_lcr.
15:18.26fileso your issue is that a native bridge isn't done with DAHDI when DTMF is required?
15:18.51WIMPyNo, that the core doesn't seem to support it any more.
15:19.17fileI don't understand what you mean
15:19.20*** join/#asterisk saint_ (~saint_@unaffiliated/saint-/x-0540772)
15:19.32WIMPyLooks like the channel starves after a digit is detected. But I'm not that deep in to it.
15:19.55filethe Asterisk core doesn't care, it's up to the native bridge logic - in fact if INFO DTMF is in use with SIP then it'll reinvite media regardless and DTMF will still go through Asterisk and work
15:20.20WIMPyWell, in 11 it doesn't work any more. And due to the comment in chan_dahdi, I know that it's something in the core.
15:21.37WIMPyIf you don't disable the netive bridge, the bridge will be broken after a digit has been detected (which is expected, I think), but it won't start again until 1. another digit is received or 2. a
15:21.41WIMPyrandom amount of time.
15:22.09fileit is expected, that's how it works in 11
15:22.20saint_hey Bordr did you figure your issue out ?
15:22.44WIMPyWell, yeah, doesn't work.
15:24.27zambadoes the order in which you specify codecs in the allow statement matter?
15:24.38zambaor will it just negotiate the best possible?
15:24.39WIMPyyes
15:24.52zambahehe, sorry.. "yes" to what? :)
15:25.03WIMPyOrder does matter
15:25.34fileWIMPy, the logic to not native bridge if DTMF is required from either side also exists in 1.8...
15:26.01WIMPyInteresting. It used to work back then.
15:26.18zambaWIMPy: is it enough to just do 'sip reload' after making the change to the sip.conf or do the devices need to re-register?
15:26.36filece7a1e4768      (Kevin P. Fleming       2005-08-09 01:59:59 +0000       7261)   /* For now, don't attempt to native bridge if either channel needs DTMF detection.
15:26.45filecode has been there for over 10 years
15:26.58WIMPyNo need to re-register, but sip reload can be alittle hit or miss when changing things.
15:27.10WIMPyWow. That's old.
15:27.23aursfeeling old now WIMPy ? :D
15:27.27zambaWIMPy: so what do i need to do to make sure that the changes take effect?
15:28.08WIMPyI thik it still worked in 10. But I'm not sure. Can't change versions all day to find out when things broke.
15:28.39WIMPyaurs: that might even have been before I first tried Asterisk.
15:28.40fileif there was a systemic problem for the user base with DAHDI in such a situation we'd likely see a trend and would jump on it, but I haven't seen anything
15:29.20*** join/#asterisk putnopvut (putnopvut@asterisk/master-of-queues/mmichelson)
15:29.21*** mode/#asterisk [+o putnopvut] by ChanServ
15:29.46WIMPyWell, chan_lcr contains te same code there that chn_dahdi disabled. But it did work before 11.
15:30.10fileunless it happens with chan_dahdi on an unmodified Asterisk, not much I can do
15:31.49zambaok, so i have g722 between polycom phones registered directly on the asterisk server, but not when crossing the lync trunk
15:31.51WIMPyCertainly interesting that cahn_dahdi had that feature disabled for ages.
15:32.47zamba'sip show channels' with a call over the lync trunk reveals alaw as the codec used
15:33.20WIMPyBut the fact that asterisk loses digits when using dundi switches is what keeps me from making calls with Asterisk.
15:33.43[TK]D-Fenderzamba: And we see no proof about what either side actually offered.
15:33.54zamba[TK]D-Fender: sorry.. i'm a bit new to this.. you mean sip debug?
15:34.17WIMPyWhat way is the call going? To or from lync?
15:34.27zambaWIMPy: from lync
15:34.27[TK]D-Fenderzamba: Obviously.
15:35.01WIMPyWell, then lync will probably only offer the CODEC it received the call with.
15:35.50zambawhat am i looking for in the debugging? rtpmap?
15:36.12*** join/#asterisk defsdoor (~andy@cpc73037-sutt4-2-0-cust62.19-1.cable.virginm.net)
15:36.22WIMPyNo need to debug that part. Where does lync get the call from?
15:36.30zambaWIMPy: a lync client
15:36.44WIMPyWhat CODECs does that support?
15:38.18zambaCapabilities: us - (alaw|g722), peer - audio=(ulaw|alaw)/video=(nothing)/text=(nothing), combined - (alaw)
15:38.20zambais this a hint?
15:38.36[TK]D-FenderClearly
15:38.45[TK]D-Fenderus - (alaw|g722) <- YOU
15:38.55[TK]D-Fender<PROTECTED>
15:38.55zambahttp://blog.schertz.name/2014/03/media-codecs-in-lync-2013/
15:39.26zambai see G.722 mentioned there.. and the phrasing "... which Lync will use in a few scenarios."
15:42.49[TK]D-FenderSomehow I think you are responsible for configuring what it is supposed to offer.  That is itsn't magically chosen
15:48.47*** join/#asterisk marceloamorim (~marcelo@189-90-192-72.isimples.com.br)
15:58.24*** join/#asterisk pbaines (~peter@188-39-51-2.static.enta.net)
16:04.39*** join/#asterisk jamesc (~kvirc@mail2.inetplc.com)
16:06.02*** join/#asterisk azerus (~badass@unaffiliated/badass)
16:06.08jamescis it possible to hangup an inbound call after a certain timeout value. I.e the dial command is not used and audio is played to the caller for example
16:06.26[TK]D-Fender"core show function TIMEOUT"
16:07.19jamescah I did not realise it could be access as a funcation thaks
16:08.00*** join/#asterisk azerus (~badass@unaffiliated/badass)
16:08.04[TK]D-Fender"core show applications"
16:08.09[TK]D-Fender"core show FUNCTIONS"
16:08.13[TK]D-FenderAnd read them all.
16:08.33[TK]D-FenderEveryone should know the basic things the dialplan can do
16:20.44*** join/#asterisk hehol (~hehol@gatekeeper.loca.net)
16:31.06*** join/#asterisk elguero (cf9a22a3@gateway/web/freenode/ip.207.154.34.163)
16:38.14*** join/#asterisk rmudgett (rmudgett@nat/digium/x-ynzgfaaviuvrbqyh)
16:47.48*** join/#asterisk mcargile (~mikec@rrcs-97-76-33-146.se.biz.rr.com)
16:56.47*** join/#asterisk mcargile (~mikec@rrcs-97-76-33-146.se.biz.rr.com)
17:00.26*** join/#asterisk vader- (~Adium@50.232.174.194)
17:07.46*** join/#asterisk C1ph3r (~C1ph3r@191.33.232.168)
17:13.44Kunsihm, having asterisk behind NAT, voice in local network is fine, when using other than lan connection, audio phone -> asterisk works, asterisk -> phone doesn't work. i've enabled ports tcp 5061, udp 5060 for sips/sip, and set udp 5004-5200 as rtp ports, also enabled in firewall. am i missing something? phone is using SIPS, asterisk 13.2.0, sip transport, not pjsip.
17:20.25ChkDigitKunsi: You probably need to flush /proc/sys/net/netfilter/nf_conntrack
17:21.14Kunsihm?
17:22.18ChkDigitThe SIP devices try to connect/register before other links are up, and then the gateway keeps a conntrack entry for NAT with the wrong route.
17:22.22WIMPyUsing one of the RTP ports for SIP does not look like a good idea.
17:23.07*** join/#asterisk Aboba (~Bob@201-085.camosun.bc.ca)
17:26.30KunsiWIMPy: hm, why didn't i see that? however, reconfigured, no change.
17:27.03KunsiChkDigit: my * is the nat gateway, and devices are registered with correct ip address
17:28.03WIMPyAre those external devices behind NAT?
17:28.52Kunsiuhm, don't really know ... it's a vodafone germany android phone with csipsimple installed
17:29.27WIMPyThat most probably makes it a yes.
17:29.46WIMPyDo you have your Asterisk configured to support peers behind NAT?
17:29.54ChkDigitWhat is the popular "pastebin" to use these days?
17:30.19*** join/#asterisk mcargile (~mikec@208.38.149.182)
17:31.01Kunsii don't think so, atleast i have nothing in sip.conf which looks like a net specific setting
17:31.24KunsiChkDigit: paste.debian.net for me
17:32.54ChkDigitKunsi: I just went ahead and used .com...  See http://pastebin.com/Y2uSs56P for a script my old company used to fix our issue, and see if yours goes away.
17:34.09ChkDigitIt is basically clearing out the UDP conntrack table NAT uses to know which external UDP connections go to which internal addresses.
17:34.14*** join/#asterisk F2Knight (~F2Knight@c-50-139-86-39.hsd1.or.comcast.net)
17:35.09ChkDigitIt is not an Asterisk problem, it is because SIP devices trying to register are very chatty, and they don't let the conntrack entry expire before trying again.
17:35.44ChkDigitSo, while the new route exists, conntrack keeps it going out over the wrong device.
17:36.30KunsiWIMPy: thank you, setting "nat=force_rport,comedia" fixed it
17:36.34[TK]D-FenderWe've seen nothing so assuming conntrack is ridiculous
17:37.17ChkDigitmeh, I put it out there because I see it all the time on VPN links.
17:37.41ChkDigitAnd Kunsi's problem had similar criteria.
17:38.08[TK]D-Fender"audio doesn't work" is not a valid assumption to jump there with
17:38.22[TK]D-FenderWe didn't see any of the basic settings he made
17:38.30ChkDigit"when using other than lan connection" was my similar.
17:38.32[TK]D-FenderAll we heard were port ranges
17:38.48[TK]D-FenderChkDigit"when using other than lan connection" was my similar. <- whichis is EVERY NAT scenario
17:39.06ChkDigitYes!  I'm glad you agree!
17:39.26[TK]D-FenderYes we didn't start with basic peer and general settings
17:40.27ChkDigitstops swinging epeens.
17:42.18*** join/#asterisk vader- (~Adium@50.232.174.194)
17:47.35*** join/#asterisk weissbier (~weissbier@2001-4ba0-ffa4-23d-0-0-0-1002.ipv6dyn.netcologne.de)
17:47.38weissbierhi
17:48.41WIMPyIs Cologne far enough south for Weissbier?
17:48.44*** join/#asterisk youtmon (~yout@c-73-46-212-142.hsd1.fl.comcast.net)
17:48.55Kunsidon't think so
17:49.11weissbierdoes pjsip stores the used codecs in any variables i could check with NoOp?
17:49.46weissbierhehe, no, not really :P
17:50.51WIMPyCheck teh CHANNEL function.
17:51.46*** join/#asterisk italorossi (~Adium@187.60.66.11)
17:51.52weissbier"audioreadformat", "audiowriteformat" seems right, cool, thanks!
17:57.31weissbierstrange... i get this: NoOp("PJSIP/sipgate-00000196", "Codecs used: g722 g722 (g722|g726|g726aal2|alaw|ulaw|gsm)") in new stack; but actually the channel uses alaw
17:57.43weissbierand the native formats looks like this: NativeFormats: (g726|alaw|ulaw|gsm)
18:14.30*** join/#asterisk C1ph3r (~C1ph3r@2804:7f3:280:1b85:bd7a:41be:4fc:a994)
18:18.10*** join/#asterisk luckman212 (~luckman21@unaffiliated/luckman212)
18:24.52*** join/#asterisk sommarnatt (~sommarnat@46-236-90-213.customer.t3.se)
18:25.31*** part/#asterisk weissbier (~weissbier@2001-4ba0-ffa4-23d-0-0-0-1002.ipv6dyn.netcologne.de)
18:40.56*** join/#asterisk Aamit (~Amit@203.187.233.9)
18:42.19Aamit<PROTECTED>
18:42.51[TK]D-FenderShow us why it should
18:42.56[TK]D-Fender~pb
18:42.57infobotA "pastebin" is a web-based service where you should paste anything over 3 lines so you don't flood the channel. Here are links to a few: http://www.pastebin.com, http://pastebin.ca, http://channels.debian.net/paste, http://paste.lisp.org, http://bin.cakephp.org/; or install pastebinit with yum or aptitude.
18:42.58[TK]D-Fender^^^
18:44.25AamitI am not getting mwi light on :(
18:44.50[TK]D-Fender[13:42][TK]D-FenderShow us why it should <-
18:45.25[TK]D-FenderShow us voicemail existing in a box's folder.  That a peer is actually looking at that mailbox
18:45.27[TK]D-Fenderetc
18:48.16*** join/#asterisk donwilliam (~donwillia@188.228.46.39)
18:55.19*** join/#asterisk sparetire_ (~sparetire@unaffiliated/sparetire)
18:58.35*** join/#asterisk goddva (~goddva@2a02:fe0:c510:e300:ae9e:17ff:fe91:a8c9)
19:02.19*** join/#asterisk ipengineer (~zconkle@static-71-252-134-63.dllstx.fios.verizon.net)
19:04.29*** join/#asterisk spicyramen (~Adium@216.239.45.89)
19:08.56*** join/#asterisk fvox13 (~stevensmi@cpe-74-74-226-116.rochester.res.rr.com)
19:21.52*** join/#asterisk twanny796 (~user@85.232.210.60)
19:32.28*** join/#asterisk jameswf_ (uid27319@gateway/web/irccloud.com/x-tzwceqyyenoliqsu)
19:35.20*** join/#asterisk GameGamer43_ (sid5533@gateway/web/irccloud.com/x-eraybkuktrhnqxca)
19:40.38*** join/#asterisk troyt (~troyt@2601:681:4600:3381:44dd:acff:fe85:9c8e)
19:44.53*** join/#asterisk ipengineer (~zconkle@static-71-252-134-63.dllstx.fios.verizon.net)
19:50.12*** join/#asterisk jonno11 (~Jon@82.163.128.49)
20:06.46*** join/#asterisk tzafrir (~tzafrir@bzq-179-40-172.cust.bezeqint.net)
20:27.00*** join/#asterisk malcolmd (malcolmd@pdpc/sponsor/digium/malcolmd)
20:27.01*** mode/#asterisk [+o malcolmd] by ChanServ
20:31.28*** join/#asterisk aross42 (~aross@OTWAON1140W-LP140-01-3096518203.dsl.bell.ca)
20:41.25*** join/#asterisk spicyramen (~Adium@216.239.45.89)
20:46.27*** join/#asterisk boson (~boson@2605:a000:fc44:5740:a800:acff:fe10:813b)
20:48.13*** join/#asterisk aross42 (~aross@OTWAON1140W-LP140-01-3096518203.dsl.bell.ca)
20:50.17*** join/#asterisk youtmon (~yout@c-73-46-212-142.hsd1.fl.comcast.net)
20:51.40*** join/#asterisk link0 (~dennisdeg@backend0.link0.net)
20:56.45*** join/#asterisk sommarnatt (~sommarnat@46-236-90-213.customer.t3.se)
21:08.17*** join/#asterisk zapata (~zapata@2a02:b18:581:10:f8d3:a159:ee88:ecec)
21:13.00*** join/#asterisk acidfoo (45a5a03c@gateway/web/freenode/ip.69.165.160.60)
21:13.17acidfoothere is a command to see the timing sources, I forgot, anyone know ? :) thank you
21:14.16pahi, question: is it easy to use external FXOs over SIP in asterisk? like the audiocodes mediapack ones
21:15.05Kunsiexternal FXO is just a sip client to asterisk
21:15.59paKunsi, ah.. so if i want to call landlines with it from asterisk what would i do? just ring that sip client and pass the number to dial?
21:17.25Kunsii just use _XX.,1,Dial(SIP/${EXTEN}@spa3102-pstn) to call POTS
21:18.41paaha interesting. thanks a lot! :)
21:19.06paKunsi, do you get also the caller ID when you receive calls from it to asterisk?
21:19.14Kunsii do
21:19.18pafantastic :)
21:19.41pathen i guess it's time to get rid of the ISDN TA, and go back to PSTN :-)
21:19.50*** join/#asterisk Tim_Toady (~fuzzy@snf-33276.vm.okeanos.grnet.gr)
21:26.11[TK]D-FenderISDN is a PSTN tech....
21:39.13*** join/#asterisk F2Knight (~F2Knight@c-50-139-86-39.hsd1.or.comcast.net)
21:51.33*** join/#asterisk boson (~boson@2605:a000:fc44:5740:a800:acff:fe10:813b)
21:51.33*** join/#asterisk jrrose (jon@nat/digium/x-atzgjfajtnrtykad)
21:53.25*** join/#asterisk marlinc (~marlinc@unaffiliated/marlinc)
21:53.58*** join/#asterisk [TK]D-Fender (~joe@64.235.216.2)
21:58.02malcolmdacidfoo: module show like res_timing_   should show you what timing modules are loaded.  the one w/ a 1 for use count is in use.
22:00.02acidfooah great thank you :)
22:02.03*** join/#asterisk Echo6 (~Echo6@64.136.247.50)
22:22.30*** join/#asterisk spicyramen (~Adium@216.239.45.89)
22:34.27*** join/#asterisk [NC] (~nc@rv1.sabius.net)
22:44.43*** join/#asterisk [NC] (~nc@rv1.sabius.net)
22:55.45*** join/#asterisk KNERD (~KNERD@netservisity.com)
23:13.14*** join/#asterisk aross42 (~aross@192-0-133-86.cpe.teksavvy.com)
23:40.48*** join/#asterisk pppingme (~pppingme@unaffiliated/pppingme)
23:44.31*** join/#asterisk youtmon (~yout@c-73-46-212-142.hsd1.fl.comcast.net)

Generated by irclog2html.pl Modified by Tim Riker to work with infobot.