Kimvais's questions

I have a stream of events happening in the new CW Event Bridge that I need to get some statistics on from a REST API endpoint, a lambda function (python + pandas).

What is the best AWS service to store this data so that it's quick to load in a lambda function?

Here are the biggest cons of the things I've considered:

• CloudWatch logs is too slow
• CloudWatch insights only gives you last 1000 without pagination (and also is too slow)
• extended retention only covers last 7 days
• S3 does not have "append" operation
• DynamoDB will (probably) consume too much write capacity
• ElasticSearch seems a bit overkill (and requires an running cluster != serverless)

I have a FreeBSD (10) host connected to a Linux (Debian 7) box directly with a Cat 6 cable.

Both hosts have Intel X-540 10GBe NICs (ixbge).

Whenever there is heavy traffic (high PPS) from the FreeBSD side, the Link state between the adapters go down. Setting FC (Autoneg, RX/TX, all) off seems to not have any effect on this.

Neither OS kernels log anything useful, only message is about <interface>: NIC Link is Down After a half a second or so, the links goes back up.

1. What might be the cause? or,
2. how to troubleshoot this?

EDIT:

Tested with packet generator: It seems that it is exactly 8188 packets that get through and 8189th causes the link go down, regardless of packet size

EDIT 2: This seems to be the transmitting end (FreeBSD) that is to blame, connecting the port in question to another port on the same host yields the exact same results.

I am using sssd to authenticate and identify users from AD using Kerberos and LDAP, however, I have to add auth/session/account sufficient pam-sss.so to /etc/pam.d/common-* files even if I check SSS authentication in pam-auth-update. Is this a bug in the package or am I missing some crucial step?

Despite multiple questions already about this, e.g. Linux on Windows AD Domain I'd like to know how to get Debian 6.0 Squeeze integrated with AD using open source or otherwise free of charge for commercial use tools only

EDIT: Only tools that have their (security) updates delivered via apt are acceptable.

So far I have been able to get the actual user authentication work via kerberos, e.g. the logs show that the username/password check succeeds, but the user is not able to login, see log excerpt below;

EDIT: Log updated with pam debug on:

May 12 10:06:33 debian-6-master login[10601]: pam_krb5(login:auth): pam_sm_authenticate: entry (0x0)
May 12 10:06:33 debian-6-master login[10601]: pam_krb5(login:auth): (user test.linux) attempting authentication as [email protected]
May 12 10:06:36 debian-6-master login[10601]: pam_krb5(login:auth): user test.linux authenticated as [email protected]
May 12 10:06:36 debian-6-master login[10601]: pam_krb5(login:auth): pam_sm_authenticate: exit (success)
May 12 10:06:36 debian-6-master login[10601]: pam_unix(login:account): could not identify user (from getpwnam(test.linux))
May 12 10:06:36 debian-6-master login[10601]: pam_krb5(login:account): pam_sm_acct_mgmt: entry (0x0)
May 12 10:06:36 debian-6-master login[10601]: pam_krb5(login:account): (user test.linux) retrieving principal from cache
May 12 10:06:36 debian-6-master login[10601]: pam_krb5(login:account): pam_sm_acct_mgmt: exit (success)
May 12 10:06:36 debian-6-master login[10601]: pam_env(login:session): No such user!?
May 12 10:06:36 debian-6-master login[10601]: pam_env(login:session): No such user!?
May 12 10:06:36 debian-6-master login[10601]: pam_krb5(login:session): pam_sm_open_session: entry (0x0)
May 12 10:06:36 debian-6-master login[10601]: pam_krb5(login:session): (user test.linux) getpwnam failed for test.linux
May 12 10:06:36 debian-6-master login[10601]: pam_krb5(login:session): pam_sm_open_session: exit (failure)
May 12 10:06:36 debian-6-master login[10601]: pam_unix(login:session): session opened for user test.linux by LOGIN(uid=0)
May 12 10:06:36 debian-6-master login[10601]: User not known to the underlying authentication module
May 12 10:06:36 debian-6-master login[10601]: PAM 1 more authentication failure; logname=LOGIN uid=0 euid=0 tty=/dev/tty3 ruser= rhost=

My ldap.conf looks like this:

base dc=ad,dc=domain
uri ldap://10.10.10.10
ldap_version 3
binddn [email protected]
bindpw password
scope sub
pam_password ad
nss_base_passwd dc=ad,dc=domain?sub
nss_base_shadow dc=ad,dc=domain?sub
nss_base_group dc=ad,dc=domain?sub? &(objectCategory=group)(gidnumber=*)
nss_map_objectclass posixAccount user
nss_map_objectclass shadowAccount user
nss_map_objectclass posixGroup group
nss_map_attribute gecos cn
nss_map_attribute homeDirectory unixHomeDirectory
nss_map_attribute uniqueMember member
pam_sasl_mech DIGEST-MD5

nsswitch.conf:

# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, try:
# `info libc "Name Service Switch"' for information about this file.

passwd:         compat
group:          compat
shadow:         compat

hosts:          files dns ldap
networks:       files ldap

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

netgroup:       nis ldap

passwd_compat: files ldap
group_compat: files ldap
shadow_compat: files ldap

all /etc/pam.d are as created by pam-auth-update, all three (Kerberos, Unix and LDAP) authentication methods selected.

I can confirm from packet capture that the LDAP search results to correct user information, the same as shown below as a manual ldapsearch result:

dn: CN=Linux\, test,OU=SpecialAccounts,OU=FI1-Helsinki,OU=EMEA,OU=_Managed Are
 as,DC=ad,DC=domain
objectClass: top
objectClass: person
objectClass: domainanizationalPerson
objectClass: user
cn: Linux, test
sn: Linux
givenName: test
distinguishedName: CN=Linux\, test,OU=SpecialAccounts,OU=FI1-Helsinki,OU=EMEA,
 OU=_Managed Areas,DC=ad,DC=domain
instanceType: 4
whenCreated: 20110407131914.0Z
whenChanged: 20110511125854.0Z
displayName: Linux, test
uSNCreated: 4144737
uSNChanged: 4638378
name: Linux, test
objectGUID:: wwZt/MX/K0S36BL4bS2w+g==
userAccountControl: 66048
badPwdCount: 0
codePage: 0
countryCode: 0
badPasswordTime: 129489044965699903
lastLogoff: 0
lastLogon: 129495915807176914
pwdLastSet: 129466559550934238
primaryGroupID: 513
objectSid:: AQUAAAAAAAUVAAAAzXxBZqg31mUH5TsrkisAAA==
accountExpires: 9223372036854775807
logonCount: 35
sAMAccountName: test.linux
sAMAccountType: 805306368
userPrincipalName: [email protected]
lockoutTime: 0
objectCategory: CN=Person,CN=Schema,CN=Configuration,DC=ad,DC=domain
dSCorePropagationData: 20110407131916.0Z
dSCorePropagationData: 16010101000000.0Z
lastLogonTimestamp: 129488989872488561
uid: test.linux
msSFU30Name: test.linux
msSFU30NisDomain: ad
uidNumber: 10002
gidNumber: 10000
unixHomeDirectory: /home/test.linux
loginShell: /bin/sh

# refldap://DomainDnsZones.ad.domain/DC=DomainDnsZones,DC=ad,DC=domain

# refldap://ForestDnsZones.ad.domain/DC=ForestDnsZones,DC=ad,DC=domain

# refldap://ad.domain/CN=Configuration,DC=ad,DC=domain

# pagedresultscookie=

1. With a correct username and password I get MOTD and a message User not known to the underlying authentication module
2. With a wrong username I get Login incorrect
3. With a correct username, but wrong password, I get SASL/DIGEST-MD5 authentication started followed by Login incorrect

The AD is running Windows 2k8(r2) server, all debian packages are what you get from apt.

Any ideas very much welcome.

EDIT 2: As suggested below, I tried with sssd with similar results, now the password is asked twice, and the log shows:

May 12 14:53:06 debian-6-master login[11389]: pam_sss(login:auth): authentication failure; logname=LOGIN uid=0 euid=0 tty=/dev/tty3 ruser= rhost= user=test.linux
May 12 14:53:06 debian-6-master login[11389]: pam_sss(login:auth): received for user test.linux: 10 (User not known to the underlying authentication module)
May 12 14:53:14 debian-6-master login[11389]: pam_krb5(login:auth): user test.linux authenticated as [email protected]
May 12 14:53:14 debian-6-master login[11389]: pam_unix(login:account): could not identify user (from getpwnam(test.linux))
May 12 14:53:15 debian-6-master login[11389]: pam_sss(login:account): Access denied for user test.linux: 10 (User not known to the underlying authentication module)
May 12 14:53:15 debian-6-master login[11389]: pam_env(login:session): No such user!?
May 12 14:53:15 debian-6-master login[11389]: pam_env(login:session): No such user!?
May 12 14:53:15 debian-6-master login[11389]: pam_krb5(login:session): (user test.linux) getpwnam failed for test.linux
May 12 14:53:15 debian-6-master login[11389]: pam_unix(login:session): session opened for user test.linux by LOGIN(uid=0)
May 12 14:53:15 debian-6-master login[11389]: User not known to the underlying authentication module

EDIT 3:

If I run sssd on the foreground with debug level set to 5, the log shows:

(Fri May 13 13:50:33 2011) [sssd[nss]] [nss_cmd_endpwent] (4): Terminating request info for all accounts
(Fri May 13 13:50:33 2011) [sssd[nss]] [nss_cmd_getpwnam] (4): Requesting info for [test.linux] from [<ALL>]
(Fri May 13 13:50:33 2011) [sssd[nss]] [nss_cmd_getpwnam] (2): No matching domain found for [test.linux], fail!
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_cmd_authenticate] (4): entering pam_cmd_authenticate
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_print_data] (4): command: PAM_AUTHENTICATE
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_print_data] (4): domain: (null)
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_print_data] (4): user: test.linux
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_print_data] (4): service: login
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_print_data] (4): tty: /dev/tty3
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_print_data] (4): ruser: (null)
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_print_data] (4): rhost: (null)
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_print_data] (4): authtok type: 1
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_print_data] (4): authtok size: 8
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_print_data] (4): newauthtok type: 0
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_print_data] (4): newauthtok size: 0
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_print_data] (4): priv: 1
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_print_data] (4): cli_pid: 12507
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_reply] (4): pam_reply get called.
(Fri May 13 13:50:34 2011) [sssd[pam]] [pam_reply] (4): blen: 8
(Fri May 13 13:50:34 2011) [sssd[nss]] [nss_cmd_endpwent] (4): Terminating request info for all accounts
(Fri May 13 13:50:34 2011) [sssd[nss]] [nss_cmd_getpwnam] (4): Requesting info for [test.linux] from [<ALL>]
(Fri May 13 13:50:34 2011) [sssd[nss]] [nss_cmd_getpwnam] (2): No matching domain found for [test.linux], fail!

I managed to get the Debian 6.0.1 to work with the Microsoft Hyper-V synthetic drivers, thanks to this question. However, if I don't remove the entry for the synthetic ethernet card from /etc/udev/rules.d/70-persistent-net.rules on every boot, the VM won't boot the next time as there is a kernel BUG at mm/mmap.c:2129 (Recursive fault) - last sysfs file: /sys/devices/vmbus_0_0/vmbus_0_3/net/eth0/uevent

How do I fix this?

EDIT:

75-persistent-net-generator.rules looks like this:

# These rules generate rules to keep network interface names unchanged
# across reboots and write them to /etc/udev/rules.d/70-persistent-net.rules.

# variables used to communicate:
#   MATCHADDR           MAC address used for the match
#   MATCHID             bus_id used for the match
#   MATCHDRV            driver name used for the match
#   MATCHIFTYPE         interface type match
#   COMMENT             comment to add to the generated rule
#   INTERFACE_NAME      requested name supplied by external tool
#   INTERFACE_NEW       new interface name returned by rule writer

ACTION!="add",                          GOTO="persistent_net_generator_end"
SUBSYSTEM!="net",                       GOTO="persistent_net_generator_end"

# ignore the interface if a name has already been set
NAME=="?*",                             GOTO="persistent_net_generator_end"

# device name whitelist
KERNEL!="eth*|ath*|wlan*[0-9]|msh*|ra*|sta*|ctc*|lcs*|hsi*", \
                                        GOTO="persistent_net_generator_end"

# ignore Xen virtual interfaces
SUBSYSTEMS=="xen",                      GOTO="persistent_net_generator_end"

# ignore UML virtual interfaces
DRIVERS=="uml-netdev",                  GOTO="persistent_net_generator_end"

# ignore "secondary" raw interfaces of the madwifi driver
KERNEL=="ath*", ATTRS{type}=="802",     GOTO="persistent_net_generator_end"

# ignore "secondary" monitor interfaces of mac80211 drivers
KERNEL=="wlan*", ATTRS{type}=="803",    GOTO="persistent_net_generator_end"

# by default match on the MAC address and interface type
ENV{MATCHADDR}="$attr{address}"
ENV{MATCHIFTYPE}="$attr{type}"

# match interface dev_id
ATTR{dev_id}=="?*", ENV{MATCHDEVID}="$attr{dev_id}"

# These vendors are known to violate the local MAC address assignment scheme
# Interlan, DEC (UNIBUS or QBUS), Apollo, Cisco, Racal-Datacom
ENV{MATCHADDR}=="02:07:01:*", GOTO="globally_administered_whitelist"
# 3Com
ENV{MATCHADDR}=="02:60:60:*", GOTO="globally_administered_whitelist"
# 3Com IBM PC; Imagen; Valid; Cisco; Apple
ENV{MATCHADDR}=="02:60:8c:*", GOTO="globally_administered_whitelist"
# Intel
ENV{MATCHADDR}=="02:a0:c9:*", GOTO="globally_administered_whitelist"
# Olivetti
ENV{MATCHADDR}=="02:aa:3c:*", GOTO="globally_administered_whitelist"
# CMC Masscomp; Silicon Graphics; Prime EXL
ENV{MATCHADDR}=="02:cf:1f:*", GOTO="globally_administered_whitelist"
# Prominet Corporation Gigabit Ethernet Switch
ENV{MATCHADDR}=="02:e0:3b:*", GOTO="globally_administered_whitelist"
# BTI (Bus-Tech, Inc.) IBM Mainframes
ENV{MATCHADDR}=="02:e6:d3:*", GOTO="globally_administered_whitelist"
# Realtek
ENV{MATCHADDR}=="52:54:00:*", GOTO="globally_administered_whitelist"
# Novell 2000
ENV{MATCHADDR}=="52:54:4c:*", GOTO="globally_administered_whitelist"
# Realtec
ENV{MATCHADDR}=="52:54:ab:*", GOTO="globally_administered_whitelist"
# Kingston Technologies
ENV{MATCHADDR}=="e2:0c:0f:*", GOTO="globally_administered_whitelist"

# ignore interfaces with locally administered or null MAC addresses
# and KVM and VMWare virtual interfaces
ENV{MATCHADDR}=="?[2367abef]:*",        ENV{MATCHADDR}=""
ENV{MATCHADDR}=="00:00:00:00:00:00",    ENV{MATCHADDR}=""
ENV{MATCHADDR}=="00:0c:29:*|00:50:56:*", ENV{MATCHADDR}=""
ENV{MATCHADDR}=="52:54:00:*|54:52:00:*", ENV{MATCHADDR}=""

LABEL="globally_administered_whitelist"

# ibmveth interfaces have stable locally administered MAC addresses
SUBSYSTEMS=="ibmveth",                  ENV{MATCHADDR}="$attr{address}"

# S/390 interfaces are matched only by id
#SUBSYSTEMS=="ccwgroup", \
#       ENV{MATCHDRV}="$driver", ENV{MATCHID}="$id", ENV{MATCHADDR}=""

# terminate processing if there are not enough conditions to create a rule
ENV{MATCHADDR}=="", ENV{MATCHID}=="", ENV{INTERFACE_NAME}=="", \
                                        GOTO="persistent_net_generator_end"


# provide nice comments for the generated rules
SUBSYSTEMS=="pci", \
 ENV{COMMENT}="PCI device $attr{vendor}:$attr{device}"
SUBSYSTEMS=="pcmcia", \
 ENV{COMMENT}="PCMCIA device $attr{card_id}:$attr{manf_id}"
SUBSYSTEMS=="usb", \
 ENV{COMMENT}="USB device 0x$attr{idVendor}:0x$attr{idProduct}"
SUBSYSTEMS=="ccwgroup", \
 ENV{COMMENT}="S/390 device at $id"
SUBSYSTEMS=="ibmveth", \
 ENV{COMMENT}="LPAR virtual device at $id"
SUBSYSTEMS=="ieee1394", \
 ENV{COMMENT}="Firewire device $attr{host_id}"
ENV{COMMENT}=="", \
 ENV{COMMENT}="Unknown $env{SUBSYSTEM} device ($env{DEVPATH})"
ATTRS{driver}=="?*", \
 ENV{COMMENT}="$env{COMMENT} ($attr{driver})"


# ignore interfaces without a driver link like bridges and VLANs, otherwise
# generate and write the rule
DRIVERS=="?*", IMPORT{program}="write_net_rules"

# rename the interface if requested
ENV{INTERFACE_NEW}=="?*", NAME="$env{INTERFACE_NEW}"

LABEL="persistent_net_generator_end"

Currently I have a /etc/postfix/virtual file that has entries like:

[email protected] [email protected] 

I want to be able to forward [email protected] to [email protected]

How do I do this?

EDIT: I did set

propagate_unmatched_extensions = canonical, virtual

and

recipient_delimeter = +

but still keep getting Recipient address rejected: User unknown in local recipient table;

EDIT II: I made the changes suggested in @Schot's updated answer with some progress;

Now I get Recipient address rejected: User unknown in virtual alias table; when sending to [email protected]

Sending to [email protected] worked both before and after them modifications

EDIT III:

Postfix is version 2.3.3 - I do not have separate mail.warn or mail.err logs - just the /var/log/maillog. Here is the full (anonymized) snippet:

EDIT IV: Updated to postfix 2.3.19 with no luck.

EDIT V: Doing postconf -n showed that recipient_delimiter was misspelled in the main.cf as well as above and the delimiter was not processed at all.