[RADIATOR] Cisco NX-OS TACACS+ problems

[Alexander Hartmaier]

Hi,
our switching guys reported that their Cisco Nexus switches running NX-OS log that their can't reach the tacacs servers. This is what the troubleshooting brought up:

2013 Oct 11 08:47:37.061 sgv20s %TACACS-3-TACACS_ERROR_MESSAGE: All servers failed to respond

149) Event:E_MTS_TX, length:60, at 60683 usecs after Fri Oct 11 08:47:37 2013
[RSP] Opc:MTS_OPC_TACACS_AAA_REQ(8421), Id:0X0A287795, Ret:SUCCESS
Src:0x00000501/112, Dst:0x00000501/111, Flags:None
HA_SEQNO:0X00000000, RRtoken:0x0A287778, Sync:UNKNOWN, Payloadsize:26
Payload:
0x0000: 01 03 01 00 3b a2 66 be 00 00 00 00 00 02 00 00


150) Event:E_MTS_RX, length:60, at 46447 usecs after Fri Oct 11 08:47:37 2013
[REQ] Opc:MTS_OPC_TACACS_AAA_REQ(8421), Id:0X0A287778, Ret:SUCCESS
Src:0x00000501/111, Dst:0x00000501/0, Flags:None
HA_SEQNO:0X00000000, RRtoken:0x0A287778, Sync:UNKNOWN, Payloadsize:371
Payload:
0x0000: 01 03 0c 00 00 00 00 00 00 00 00 00 00 00 02 00


According to Cisco the accounting responses from Radiator (version 4.11 with patches revision 1.1530) contain errors:

Accounting Statistics
failed transactions: 1865
successful transactions: 0
requests sent: 1865
requests timed out: 4
responses with no matching requests: 0
responses not processed: 0
responses containing errors: 1861


Did someone else notice these problems? Authentication works without any problems.

--
Best regards, Alexander Hartmaier

T-Systems Austria GesmbH
TSS Security Services
Network Security & Monitoring Engineer

phone: [phone]
fax: [phone]



*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
Handelsgericht Wien, FN 79340b
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
Notice: This e-mail contains information that is confidential and may be privileged.
If you are not the intended recipient, please notify the sender and then
delete this e-mail immediately.
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.open.com.au/pipermail/radiator/attachments/20131011/bffd61af/attachment.html

[Caporossi, Steve G.]

We also have issues with NXOS; in our case using RADIUS.

It always seems to begin with these syslog messages;
2013 Oct 10 19:56:14.103 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
2013 Oct 10 19:56:14.105 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
2013 Oct 10 19:56:14.106 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
2013 Oct 10 19:56:14.107 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: All RADIUS servers failed to respon
d after retries.

Authentication fails and we to fallback to local authentication to "fix" the issue by sending test authentication to the RADIUS servers.

We have the DNS entries configured on the Nexus devices and when this is happening the device can ping the servers using the hostname. Another strange thing is it happens primarily in one VDC and much less frequently on the others using the same OOB management network.

Steve


On Oct 11, 2013, at 4:38 AM, Alexander Hartmaier <alexander.hartmaier at t-systems.at>
wrote:

> Hi,
> our switching guys reported that their Cisco Nexus switches running NX-OS log that their can't reach the tacacs servers. This is what the troubleshooting brought up:
>
> 2013 Oct 11 08:47:37.061 sgv20s %TACACS-3-TACACS_ERROR_MESSAGE: All servers failed to respond
>
>
> 149) Event:E_MTS_TX, length:60, at 60683 usecs after Fri Oct 11 08:47:37 2013
>
> [RSP] Opc:MTS_OPC_TACACS_AAA_REQ(8421), Id:0X0A287795, Ret:SUCCESS
>
> Src:0x00000501/112, Dst:0x00000501/111, Flags:None
>
> HA_SEQNO:0X00000000, RRtoken:0x0A287778, Sync:UNKNOWN, Payloadsize:26
>
> Payload:
>
> 0x0000: 01 03 01 00 3b a2 66 be 00 00 00 00 00 02 00 00
>
>
>
> 150) Event:E_MTS_RX, length:60, at 46447 usecs after Fri Oct 11 08:47:37 2013
>
> [REQ] Opc:MTS_OPC_TACACS_AAA_REQ(8421), Id:0X0A287778, Ret:SUCCESS
>
> Src:0x00000501/111, Dst:0x00000501/0, Flags:None
>
> HA_SEQNO:0X00000000, RRtoken:0x0A287778, Sync:UNKNOWN, Payloadsize:371
>
> Payload:
>
> 0x0000: 01 03 0c 00 00 00 00 00 00 00 00 00 00 00 02 00
>
>
> According to Cisco the accounting responses from Radiator (version 4.11 with patches revision 1.1530) contain errors:
>
> Accounting Statistics
>
> failed transactions: 1865
>
> successful transactions: 0
>
> requests sent: 1865
>
> requests timed out: 4
>
> responses with no matching requests: 0
>
> responses not processed: 0
>
> responses containing errors: 1861
>
>
> Did someone else notice these problems? Authentication works without any problems.
>
> --
> Best regards, Alexander Hartmaier
>
> T-Systems Austria GesmbH
> TSS Security Services
> Network Security & Monitoring Engineer
>
> phone: [phone]
> fax: [phone]
>
>
>
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
> T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
> Handelsgericht Wien, FN 79340b
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
> Notice: This e-mail contains information that is confidential and may be privileged.
> If you are not the intended recipient, please notify the sender and then
> delete this e-mail immediately.
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
> _______________________________________________
> radiator mailing list
> radiator at open.com.au
> http://www.open.com.au/mailman/listinfo/radiator

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4433 bytes
Desc: not available
Url : http://www.open.com.au/pipermail/radiator/attachments/20131011/b9c4dcc4/attachment-0001.bin

[Alexander Hartmaier]

On 2013-10-11 13:56, Caporossi, Steve G. wrote:
> We also have issues with NXOS; in our case using RADIUS.
>
> It always seems to begin with these syslog messages;
> 2013 Oct 10 19:56:14.103 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
> 2013 Oct 10 19:56:14.105 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
> 2013 Oct 10 19:56:14.106 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
> 2013 Oct 10 19:56:14.107 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: All RADIUS servers failed to respon
> d after retries.
>
> Authentication fails and we to fallback to local authentication to "fix" the issue by sending test authentication to the RADIUS servers.
>
> We have the DNS entries configured on the Nexus devices and when this is happening the device can ping the servers using the hostname. Another strange thing is it happens primarily in one VDC and much less frequently on the others using the same OOB management network.
What do you mean with 'dns entries configured *on* the Nexus'? Does it
happen too if you configure the radius servers ip addresses instead of
their dns names?

@Radiator guys: any update from you?

>
> Steve
>
>
> On Oct 11, 2013, at 4:38 AM, Alexander Hartmaier <alexander.hartmaier at t-systems.at>
> wrote:
>
>> Hi,
>> our switching guys reported that their Cisco Nexus switches running NX-OS log that their can't reach the tacacs servers. This is what the troubleshooting brought up:
>>
>> 2013 Oct 11 08:47:37.061 sgv20s %TACACS-3-TACACS_ERROR_MESSAGE: All servers failed to respond
>>
>>
>> 149) Event:E_MTS_TX, length:60, at 60683 usecs after Fri Oct 11 08:47:37 2013
>>
>> [RSP] Opc:MTS_OPC_TACACS_AAA_REQ(8421), Id:0X0A287795, Ret:SUCCESS
>>
>> Src:0x00000501/112, Dst:0x00000501/111, Flags:None
>>
>> HA_SEQNO:0X00000000, RRtoken:0x0A287778, Sync:UNKNOWN, Payloadsize:26
>>
>> Payload:
>>
>> 0x0000: 01 03 01 00 3b a2 66 be 00 00 00 00 00 02 00 00
>>
>>
>>
>> 150) Event:E_MTS_RX, length:60, at 46447 usecs after Fri Oct 11 08:47:37 2013
>>
>> [REQ] Opc:MTS_OPC_TACACS_AAA_REQ(8421), Id:0X0A287778, Ret:SUCCESS
>>
>> Src:0x00000501/111, Dst:0x00000501/0, Flags:None
>>
>> HA_SEQNO:0X00000000, RRtoken:0x0A287778, Sync:UNKNOWN, Payloadsize:371
>>
>> Payload:
>>
>> 0x0000: 01 03 0c 00 00 00 00 00 00 00 00 00 00 00 02 00
>>
>>
>> According to Cisco the accounting responses from Radiator (version 4.11 with patches revision 1.1530) contain errors:
>>
>> Accounting Statistics
>>
>> failed transactions: 1865
>>
>> successful transactions: 0
>>
>> requests sent: 1865
>>
>> requests timed out: 4
>>
>> responses with no matching requests: 0
>>
>> responses not processed: 0
>>
>> responses containing errors: 1861
>>
>>
>> Did someone else notice these problems? Authentication works without any problems.
>>
>> --
>> Best regards, Alexander Hartmaier
>>
>> T-Systems Austria GesmbH
>> TSS Security Services
>> Network Security & Monitoring Engineer
>>
>> phone: [phone]
>> fax: [phone]
>>
>>
>>
>> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
>> T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
>> Handelsgericht Wien, FN 79340b
>> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
>> Notice: This e-mail contains information that is confidential and may be privileged.
>> If you are not the intended recipient, please notify the sender and then
>> delete this e-mail immediately.
>> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
>> _______________________________________________
>> radiator mailing list
>> radiator at open.com.au
>> http://www.open.com.au/mailman/listinfo/radiator

[Heikki Vatiainen]

On 10/18/2013 11:23 AM, Alexander Hartmaier wrote:
> On 2013-10-11 13:56, Caporossi, Steve G. wrote:
>> We also have issues with NXOS; in our case using RADIUS.
>>
>> It always seems to begin with these syslog messages;
>> 2013 Oct 10 19:56:14.103 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
>> 2013 Oct 10 19:56:14.105 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
>> 2013 Oct 10 19:56:14.106 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
>> 2013 Oct 10 19:56:14.107 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: All RADIUS servers failed to respon
>> d after retries.
>>
>> Authentication fails and we to fallback to local authentication to "fix" the issue by sending test authentication to the RADIUS servers.
>>
>> We have the DNS entries configured on the Nexus devices and when this is happening the device can ping the servers using the hostname. Another strange thing is it happens primarily in one VDC and much less frequently on the others using the same OOB management network.
> What do you mean with 'dns entries configured *on* the Nexus'? Does it
> happen too if you configure the radius servers ip addresses instead of
> their dns names?
>
> @Radiator guys: any update from you?

For the RADIUS/DNS problem above, I can only think of configuring the
server with address instead of name. Why it fails? Maybe there's a rate
limit on the DNS side. If there are lots of RADIUS requests each causing
a DNS lookup, that might cause the lookup failures.

What comes to NX-OS problems Alexander sees, could it be possible that
accounting requests are sent to different Radiators than authentication
or authorization requests?

If so, then there might be a different shared key configured on the
NX-OS than on Radiator? In this case Radiator logs should show errors
hinting about 'Bad key?'. If Radiator thinks the key is bad, it will
disconnect and this may be logged as 'All servers failed to respond'.

Thanks,
Heikki

--
Heikki Vatiainen <hvn at open.com.au>

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald,
Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS,
TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS,
NetWare etc.

[Alexander Hartmaier]

On 2013-10-18 11:07, Heikki Vatiainen wrote:
> On 10/18/2013 11:23 AM, Alexander Hartmaier wrote:
>> On 2013-10-11 13:56, Caporossi, Steve G. wrote:
>>> We also have issues with NXOS; in our case using RADIUS.
>>>
>>> It always seems to begin with these syslog messages;
>>> 2013 Oct 10 19:56:14.103 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
>>> 2013 Oct 10 19:56:14.105 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
>>> 2013 Oct 10 19:56:14.106 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
>>> 2013 Oct 10 19:56:14.107 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: All RADIUS servers failed to respon
>>> d after retries.
>>>
>>> Authentication fails and we to fallback to local authentication to "fix" the issue by sending test authentication to the RADIUS servers.
>>>
>>> We have the DNS entries configured on the Nexus devices and when this is happening the device can ping the servers using the hostname. Another strange thing is it happens primarily in one VDC and much less frequently on the others using the same OOB management network.
>> What do you mean with 'dns entries configured *on* the Nexus'? Does it
>> happen too if you configure the radius servers ip addresses instead of
>> their dns names?
>>
>> @Radiator guys: any update from you?
> For the RADIUS/DNS problem above, I can only think of configuring the
> server with address instead of name. Why it fails? Maybe there's a rate
> limit on the DNS side. If there are lots of RADIUS requests each causing
> a DNS lookup, that might cause the lookup failures.
>
> What comes to NX-OS problems Alexander sees, could it be possible that
> accounting requests are sent to different Radiators than authentication
> or authorization requests?
>
> If so, then there might be a different shared key configured on the
> NX-OS than on Radiator? In this case Radiator logs should show errors
> hinting about 'Bad key?'. If Radiator thinks the key is bad, it will
> disconnect and this may be logged as 'All servers failed to respond'.
The requests are sent to two Radiator servers forming a faiover pair
which both have the same TACACS key.
It only happens from time to time, the authentication and accouting
requests usually work.

>
> Thanks,
> Heikki
>



*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
Handelsgericht Wien, FN 79340b
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
Notice: This e-mail contains information that is confidential and may be privileged.
If you are not the intended recipient, please notify the sender and then
delete this e-mail immediately.
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*

[Heikki Vatiainen]

On 10/18/2013 12:14 PM, Alexander Hartmaier wrote:

> The requests are sent to two Radiator servers forming a faiover pair
> which both have the same TACACS key.
> It only happens from time to time, the authentication and accouting
> requests usually work.

Have you tried with SingleSession option? This sets the
TAC_PLUS_SINGLE_CONNECT_FLAG flag as described in
http://tools.ietf.org/html/draft-grant-tacacs-02

Thanks,
Heikki

--
Heikki Vatiainen <hvn at open.com.au>

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald,
Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS,
TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS,
NetWare etc.

[Hartmaier Alexander]

On 2013-10-18 13:07, Heikki Vatiainen wrote:
> On 10/18/2013 12:14 PM, Alexander Hartmaier wrote:
>
>> The requests are sent to two Radiator servers forming a faiover pair
>> which both have the same TACACS key.
>> It only happens from time to time, the authentication and accouting
>> requests usually work.
> Have you tried with SingleSession option? This sets the
> TAC_PLUS_SINGLE_CONNECT_FLAG flag as described in
> http://tools.ietf.org/html/draft-grant-tacacs-02

That's a good pointer, thanks!
According to the Radiator ref.pdf it defaults to 1 (enabled) (we're
running 4.11 + patches), should I try to disable it? Can this be done
for some clients only too?

>
> Thanks,
> Heikki
>



*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
Handelsgericht Wien, FN 79340b
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
Notice: This e-mail contains information that is confidential and may be privileged.
If you are not the intended recipient, please notify the sender and then
delete this e-mail immediately.
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*

[Heikki Vatiainen]

On 10/18/2013 02:23 PM, Hartmaier Alexander wrote:

>> Have you tried with SingleSession option? This sets the
>> TAC_PLUS_SINGLE_CONNECT_FLAG flag as described in
>> http://tools.ietf.org/html/draft-grant-tacacs-02

> According to the Radiator ref.pdf it defaults to 1 (enabled) (we're
> running 4.11 + patches), should I try to disable it? Can this be done
> for some clients only too?

It's a server level flag but you can specify it on the client side. On
IOS something like this should do it:

tacacs-server host ... single-connection

Thanks,
Heikki

--
Heikki Vatiainen <hvn at open.com.au>

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald,
Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS,
TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS,
NetWare etc.

[Caporossi, Steve G.]

I have the host entries defined on the Nexus gear.
ip host radserver1.musc.edu <server address>
ip host radserver2.musc.edu <server address>
ip host radserver3.musc.edu <server address>

RADIUS servers *are* defined by IP address however the Nexus gears tries to resolve the hostname(s)

Steve
[phone]





On Oct 18, 2013, at 4:23 AM, Alexander Hartmaier <alexander.hartmaier at t-systems.at>
wrote:

> On 2013-10-11 13:56, Caporossi, Steve G. wrote:
>> We also have issues with NXOS; in our case using RADIUS.
>>
>> It always seems to begin with these syslog messages;
>> 2013 Oct 10 19:56:14.103 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
>> 2013 Oct 10 19:56:14.105 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
>> 2013 Oct 10 19:56:14.106 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: Failed looking up IP address for RADIUS server <server address>
>> 2013 Oct 10 19:56:14.107 mdf1 %RADIUS-3-RADIUS_ERROR_MESSAGE: All RADIUS servers failed to respon
>> d after retries.
>>
>> Authentication fails and we to fallback to local authentication to "fix" the issue by sending test authentication to the RADIUS servers.
>>
>> We have the DNS entries configured on the Nexus devices and when this is happening the device can ping the servers using the hostname. Another strange thing is it happens primarily in one VDC and much less frequently on the others using the same OOB management network.
> What do you mean with 'dns entries configured *on* the Nexus'? Does it
> happen too if you configure the radius servers ip addresses instead of
> their dns names?
>
> @Radiator guys: any update from you?
>
>>
>> Steve
>>
>>
>> On Oct 11, 2013, at 4:38 AM, Alexander Hartmaier <alexander.hartmaier at t-systems.at>
>> wrote:
>>
>>> Hi,
>>> our switching guys reported that their Cisco Nexus switches running NX-OS log that their can't reach the tacacs servers. This is what the troubleshooting brought up:
>>>
>>> 2013 Oct 11 08:47:37.061 sgv20s %TACACS-3-TACACS_ERROR_MESSAGE: All servers failed to respond
>>>
>>>
>>> 149) Event:E_MTS_TX, length:60, at 60683 usecs after Fri Oct 11 08:47:37 2013
>>>
>>> [RSP] Opc:MTS_OPC_TACACS_AAA_REQ(8421), Id:0X0A287795, Ret:SUCCESS
>>>
>>> Src:0x00000501/112, Dst:0x00000501/111, Flags:None
>>>
>>> HA_SEQNO:0X00000000, RRtoken:0x0A287778, Sync:UNKNOWN, Payloadsize:26
>>>
>>> Payload:
>>>
>>> 0x0000: 01 03 01 00 3b a2 66 be 00 00 00 00 00 02 00 00
>>>
>>>
>>>
>>> 150) Event:E_MTS_RX, length:60, at 46447 usecs after Fri Oct 11 08:47:37 2013
>>>
>>> [REQ] Opc:MTS_OPC_TACACS_AAA_REQ(8421), Id:0X0A287778, Ret:SUCCESS
>>>
>>> Src:0x00000501/111, Dst:0x00000501/0, Flags:None
>>>
>>> HA_SEQNO:0X00000000, RRtoken:0x0A287778, Sync:UNKNOWN, Payloadsize:371
>>>
>>> Payload:
>>>
>>> 0x0000: 01 03 0c 00 00 00 00 00 00 00 00 00 00 00 02 00
>>>
>>>
>>> According to Cisco the accounting responses from Radiator (version 4.11 with patches revision 1.1530) contain errors:
>>>
>>> Accounting Statistics
>>>
>>> failed transactions: 1865
>>>
>>> successful transactions: 0
>>>
>>> requests sent: 1865
>>>
>>> requests timed out: 4
>>>
>>> responses with no matching requests: 0
>>>
>>> responses not processed: 0
>>>
>>> responses containing errors: 1861
>>>
>>>
>>> Did someone else notice these problems? Authentication works without any problems.
>>>
>>> --
>>> Best regards, Alexander Hartmaier
>>>
>>> T-Systems Austria GesmbH
>>> TSS Security Services
>>> Network Security & Monitoring Engineer
>>>
>>> phone: [phone]
>>> fax: [phone]
>>>
>>>
>>>
>>> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
>>> T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
>>> Handelsgericht Wien, FN 79340b
>>> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
>>> Notice: This e-mail contains information that is confidential and may be privileged.
>>> If you are not the intended recipient, please notify the sender and then
>>> delete this e-mail immediately.
>>> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
>>> _______________________________________________
>>> radiator mailing list
>>> radiator at open.com.au
>>> http://www.open.com.au/mailman/listinfo/radiator
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4433 bytes
Desc: not available
Url : http://www.open.com.au/pipermail/radiator/attachments/20131018/312283cf/attachment-0001.bin

[Heikki Vatiainen]

On 10/18/2013 03:04 PM, Caporossi, Steve G. wrote:
> I have the host entries defined on the Nexus gear.
> ip host radserver1.musc.edu <server address>
> ip host radserver2.musc.edu <server address>
> ip host radserver3.musc.edu <server address>
>
> RADIUS servers *are* defined by IP address however the Nexus gears tries to resolve the hostname(s)

Hmm, just to clarify, you have configured hostname mappings for RADIUS
servers (ip host ...) as above, but do you mean you are using IP
addresses or names with 'radius-server host ...'?

What I'm thinking is that is it known that radius server name lookup
uses the static name to ip definitions? The cisco docs do not say if all
name lookups use the local definitions.

I do not if it does or not, since I have usually seen and used 'no ip
domain-lookup' when working with IOS. I guess this is not an option at
this point? Maybe in a lab?

Thanks,
Heikki

--
Heikki Vatiainen <hvn at open.com.au>

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald,
Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS,
TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS,
NetWare etc.

[Caporossi, Steve G.]

Steve
843.876.5083
Sent from my mobile device please excuse brevity and grammar.

> On Oct 21, 2013, at 3:43 PM, "Heikki Vatiainen" <hvn at open.com.au> wrote:
>
>> On 10/18/2013 03:04 PM, Caporossi, Steve G. wrote:
>> I have the host entries defined on the Nexus gear.
>> ip host radserver1.musc.edu <server address>
>> ip host radserver2.musc.edu <server address>
>> ip host radserver3.musc.edu <server address>
>>
>> RADIUS servers *are* defined by IP address however the Nexus gears tries to resolve the hostname(s)
>
> Hmm, just to clarify, you have configured hostname mappings for RADIUS
> servers (ip host ...) as above, but do you mean you are using IP
> addresses or names with 'radius-server host ...'?

Correct

IP addresses with radius-server host

> What I'm thinking is that is it known that radius server name lookup
> uses the static name to ip definitions?

No
> The cisco docs do not say if all
> name lookups use the local definitions.
>
> I do not if it does or not, since I have usually seen and used 'no ip
> domain-lookup' when working with IOS. I guess this is not an option at
> this point? Maybe in a lab?
>

I will disable domain-lookup and see if that resolves the issue

> Thanks,
> Heikki
>
> --
> Heikki Vatiainen <hvn at open.com.au>
>
> Radiator: the most portable, flexible and configurable RADIUS server
> anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald,
> Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS,
> TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP,
> DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS,
> NetWare etc.
> _______________________________________________
> radiator mailing list
> radiator at open.com.au
> http://www.open.com.au/mailman/listinfo/radiator

[Heikki Vatiainen]

On 10/11/2013 11:38 AM, Alexander Hartmaier wrote:

> our switching guys reported that their Cisco Nexus switches running
> NX-OS log that their can't reach the tacacs servers. This is what the
> troubleshooting brought up:
>
> 2013 Oct 11 08:47:37.061 sgv20s %TACACS-3-TACACS_ERROR_MESSAGE: All
> servers failed to respond

Returning to the subject with new information. This problem was seen by
others too and this time a fix seems to be found.

The bug appears to be CSCtz32293 and is corrected in 5.2(1)N1(5). The
upgrade was done to 5.2(1)N1(6) which shows no problems.

A similar looking problem is also described here:
http://www.cisco.com/en/US/tech/tk59/technologies_tech_note09186a0080c17808.shtml

I'm not sure if this relates to Steve's problem but looks exactly what
Alexander was seeing.

Thanks,
Heikki

--
Heikki Vatiainen <hvn at open.com.au>

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald,
Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS,
TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS,
NetWare etc.

[Hartmaier Alexander]

On 2014-02-06 23:11, Heikki Vatiainen wrote:
> On 10/11/2013 11:38 AM, Alexander Hartmaier wrote:
>
>> our switching guys reported that their Cisco Nexus switches running
>> NX-OS log that their can't reach the tacacs servers. This is what the
>> troubleshooting brought up:
>>
>> 2013 Oct 11 08:47:37.061 sgv20s %TACACS-3-TACACS_ERROR_MESSAGE: All
>> servers failed to respond
> Returning to the subject with new information. This problem was seen by
> others too and this time a fix seems to be found.
>
> The bug appears to be CSCtz32293 and is corrected in 5.2(1)N1(5). The
> upgrade was done to 5.2(1)N1(6) which shows no problems.
>
> A similar looking problem is also described here:
> http://www.cisco.com/en/US/tech/tk59/technologies_tech_note09186a0080c17808.shtml
>
> I'm not sure if this relates to Steve's problem but looks exactly what
> Alexander was seeing.
Thanks for keeping track of this problem!!!
I had no time to further investigate it with our switching guys but
informed them about the update.

>
> Thanks,
> Heikki
>



*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
Handelsgericht Wien, FN 79340b
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
Notice: This e-mail contains information that is confidential and may be privileged.
If you are not the intended recipient, please notify the sender and then
delete this e-mail immediately.
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*

[Hartmaier Alexander]

On 2014-02-07 08:35, Hartmaier Alexander wrote:
> On 2014-02-06 23:11, Heikki Vatiainen wrote:
>> On 10/11/2013 11:38 AM, Alexander Hartmaier wrote:
>>
>>> our switching guys reported that their Cisco Nexus switches running
>>> NX-OS log that their can't reach the tacacs servers. This is what the
>>> troubleshooting brought up:
>>>
>>> 2013 Oct 11 08:47:37.061 sgv20s %TACACS-3-TACACS_ERROR_MESSAGE: All
>>> servers failed to respond
>> Returning to the subject with new information. This problem was seen by
>> others too and this time a fix seems to be found.
>>
>> The bug appears to be CSCtz32293 and is corrected in 5.2(1)N1(5). The
>> upgrade was done to 5.2(1)N1(6) which shows no problems.
>>
>> A similar looking problem is also described here:
>> http://www.cisco.com/en/US/tech/tk59/technologies_tech_note09186a0080c17808.shtml
>>
>> I'm not sure if this relates to Steve's problem but looks exactly what
>> Alexander was seeing.
> Thanks for keeping track of this problem!!!
> I had no time to further investigate it with our switching guys but
> informed them about the update.
Sadly they are already running version 5.2(1)N1(6) and the error
messages still occur.
>
>> Thanks,
>> Heikki
>>
>
>
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
> T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
> Handelsgericht Wien, FN 79340b
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
> Notice: This e-mail contains information that is confidential and may be privileged.
> If you are not the intended recipient, please notify the sender and then
> delete this e-mail immediately.
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
> _______________________________________________
> radiator mailing list
> radiator at open.com.au
> http://www.open.com.au/mailman/listinfo/radiator

[Caporossi, Steve G.]

We upgraded to version 5.2(9) last weekend and our problem appears to be solved.

Thanks for keeping this on your radar.

Steve

On Feb 6, 2014, at 5:11 PM, Heikki Vatiainen <hvn at open.com.au> wrote:

> On 10/11/2013 11:38 AM, Alexander Hartmaier wrote:
>
>> our switching guys reported that their Cisco Nexus switches running
>> NX-OS log that their can't reach the tacacs servers. This is what the
>> troubleshooting brought up:
>>
>> 2013 Oct 11 08:47:37.061 sgv20s %TACACS-3-TACACS_ERROR_MESSAGE: All
>> servers failed to respond
>
> Returning to the subject with new information. This problem was seen by
> others too and this time a fix seems to be found.
>
> The bug appears to be CSCtz32293 and is corrected in 5.2(1)N1(5). The
> upgrade was done to 5.2(1)N1(6) which shows no problems.
>
> A similar looking problem is also described here:
> http://www.cisco.com/en/US/tech/tk59/technologies_tech_note09186a0080c17808.shtml
>
> I'm not sure if this relates to Steve's problem but looks exactly what
> Alexander was seeing.
>
> Thanks,
> Heikki
>
> --
> Heikki Vatiainen <hvn at open.com.au>
>
> Radiator: the most portable, flexible and configurable RADIUS server
> anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald,
> Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS,
> TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP,
> DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS,
> NetWare etc.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4433 bytes
Desc: not available
Url : http://www.open.com.au/pipermail/radiator/attachments/20140219/72f7c25a/attachment.bin

[Heikki Vatiainen]

On 02/19/2014 04:40 PM, Caporossi, Steve G. wrote:
> We upgraded to version 5.2(9) last weekend and our problem appears to be solved.
>
> Thanks for keeping this on your radar.

Good to hear. Thanks for letting us know the problem was solved.

Maybe NX-OS devices Alexander mentioned are still using a version of
NX-OS that does not have the patch? A quick look tells there are not as
many different software trains as there are/were for IOS, but there are
plenty of minor releases still.

Thanks,
Heikki

--
Heikki Vatiainen <hvn at open.com.au>

Radiator: the most portable, flexible and configurable RADIUS server
anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald,
Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS,
TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP,
DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS,
NetWare etc.