I see that ephemeral drives are now encrypted but is network communications between containers encrypted, say for HIPAA compliance. Looking specifically at Serverless Fargate with Kubernetes pods. A google of "fargate kubernetes network encryption" did not seem return relevant results, it was all about the ephemeral drives. If it is not, is there a way to do it?
Update:
I just found something that said you can set up a CNI but it wasn't 100% clear to me that that's possible with Fargate EKS.
So I'm 90 percent there, but it appears that cloudfront is getting 500 errors from s3? I'm sure I'm just doing something wrong. I've read the various amazon documentation, but to me it seems so vague as to perhaps be useless.
resource "aws_s3_bucket" "Artifacts" {
bucket = "my.domain.tld"
acl = "private"
versioning {
enabled = true
}
}
module "BucketPolicy" {
source = "../modules/S3CloudFrontBucketPolicy"
bucket_id = "${aws_s3_bucket.Artifacts.id}"
arn = "${aws_s3_bucket.Artifacts.arn}"
principal = "${module.ArtifactsCloudfront.oai_principal}"
}
module "ArtifactsCloudfront" {
source = "../modules/CloudFrontS3"
zone_id = "${aws_route53_zone.Primary.id}"
root = "/"
origin_fqdn = "${aws_s3_bucket.Artifacts.bucket_domain_name}"
user_fqdn = "${aws_s3_bucket.Artifacts.bucket}"
}
CloudFrontS3
variable "user_fqdn" {}
variable "origin_fqdn" {}
variable "zone_id" {}
variable "root" {}
output "oai_principal" {
value = "${aws_cloudfront_origin_access_identity.OAI.iam_arn}"
}
resource "aws_acm_certificate" "Cert" {
domain_name = "${var.user_fqdn}"
validation_method = "DNS"
tags {
env = "${terraform.env}"
}
}
resource "aws_route53_record" "ValidationDNS" {
name = "${aws_acm_certificate.Cert.domain_validation_options.0.resource_record_name}"
type = "${aws_acm_certificate.Cert.domain_validation_options.0.resource_record_type}"
zone_id = "${var.zone_id}"
records = ["${aws_acm_certificate.Cert.domain_validation_options.0.resource_record_value}"]
ttl = 60
}
resource "aws_acm_certificate_validation" "CertValidation" {
certificate_arn = "${aws_acm_certificate.Cert.arn}"
validation_record_fqdns = ["${aws_route53_record.ValidationDNS.fqdn}"]
}
resource "aws_route53_record" "DomainName" {
zone_id = "${var.zone_id}"
name = "${var.user_fqdn}"
type = "CNAME"
ttl = "300"
records = [
"${aws_cloudfront_distribution.Distribution.domain_name}"]
}
resource "aws_cloudfront_origin_access_identity" "OAI" {
}
resource "aws_cloudfront_distribution" "Distribution" {
aliases = ["${var.user_fqdn}"]
origin {
domain_name = "${var.origin_fqdn}"
origin_id = "${var.origin_fqdn}"
s3_origin_config {
origin_access_identity = "${aws_cloudfront_origin_access_identity.OAI.cloudfront_access_identity_path}"
}
}
enabled = true
is_ipv6_enabled = true
default_root_object = "${var.root}"
default_cache_behavior {
allowed_methods = [
"GET",
"HEAD",
"OPTIONS"]
cached_methods = [
"GET",
"HEAD",
"OPTIONS"]
forwarded_values {
cookies {
forward = "none"
}
query_string = true
}
default_ttl = 3600
max_ttl = 86400
min_ttl = 60
target_origin_id = "${var.origin_fqdn}"
viewer_protocol_policy = "https-only"
compress = true
}
price_class = "PriceClass_100"
restrictions {
geo_restriction {
restriction_type = "whitelist"
locations = ["US"]
}
}
viewer_certificate {
acm_certificate_arn = "${aws_acm_certificate_validation.CertValidation.certificate_arn}"
ssl_support_method = "sni-only"
minimum_protocol_version = "TLSv1.1_2016"
}
tags {
env = "${terraform.env}"
}
}
S3CloudFrontBucketPolicy
variable "principal" {}
variable "arn" {}
variable "bucket_id" {}
resource "aws_s3_bucket_policy" "Policy" {
bucket = "${var.bucket_id}"
policy = "${data.template_file.Policy.rendered}"
}
data "template_file" "Policy" {
vars {
arn = "${var.arn}"
oai = "${var.principal}"
}
template = <<POLICY
{
"Id": "Policy1520441044381",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1520439447147",
"Action": [
"s3:*"
],
"Effect": "Allow",
"Resource": "$${arn}/*",
"Principal": {
"AWS": [
"$${oai}"
]
}
}
]
}
POLICY
}
however when I try to actually download something I get
The request could not be satisfied. CloudFront is currently experiencing problems with requesting objects from Amazon S3.
what am I doing wrong? what do I need to fix?
I'd like one of my containers to be able to control docker itself, the way I'm currently doing it is by simply exposing the socket internally, but I don't like this. What I'd really like to do is do it over tcp, authenticated. What I'm not sure of is how I can expose the docker tcp socket (on consistent IP or Domain) to the internal private network only.
How can I expose docker as a service to my containers?
Ok the network so far is
end user (https)-> cloudfront (http)-> haproxy (http)-> jenkins
My jenkins install mostly works but I see the error about proxy not being set up correctly, and I experience https://issues.jenkins-ci.org/browse/JENKINS-14313
I've read the following, but encountered some confusing issues.
This is what I've come up with so far
defaults
mode http
retries 3
timeout connect 120s
timeout client 60s
timeout server 60s
resolvers docker
nameserver dns "127.0.0.11:53"
frontend web
bind *:8080
default_backend jenkins
backend jenkins
cookie SERVERID insert indirect nocache
server jenkins jenkins:8080 check cookie s1 resolvers docker resolve-prefer ipv4
acl h_cfp_exists req.hdr(CloudFront-Forwarded-Proto) -m found
acl response-is-redirect res.hdr(Location) -m found
http-request set-header X-Forwarded-Proto https if h_cfp_exists
Here's the headers(headers have had their domain and secure strings mangled, slashes and names preserved) received by jenkins
POST /j_acegi_security_check HTTP/1.1
Host: jenkins.default.mydomain.tld
X-Amz-Cf-Id: _UoBjzMmiJvkMZOnRbt6Ai2rBZ4hm1jAz_5IGAi3OwI9TxQ5tCW5Bg==
Content-Length: 339
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36 OPR/49.0.2725.64
Via: 2.0 9a063989aee4dbd9381a2cf612c17c08.cloudfront.net (CloudFront)
Cookie: JSESSIONID.0262fa7d=node01f8yc6mw0a5p6gv09wvy275j610.node0; JSESSIONID.9f7e10bd=node01i3tan71bopiv2ufhmos5dccq83.node0; JSESSIONID.d3f1b83b=node0nqkbin0vx70f12cdr9kmgatyk6.node0
X-Forwarded-For: 2605:6000:1025:bd:31c7:df4d:6bd6:8b9c
CloudFront-Is-Mobile-Viewer: false
CloudFront-Is-Tablet-Viewer: false
CloudFront-Is-SmartTV-Viewer: false
CloudFront-Is-Desktop-Viewer: true
CloudFront-Viewer-Country: US
Accept-Language: en-US,en;q=0.9
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Referer: https://jenkins.default.mydomain.tld/
CloudFront-Forwarded-Proto: https
Accept-Encoding: gzip, deflate, br
cache-control: max-age=0
origin: https://jenkins.default.mydomain.tld
upgrade-insecure-requests: 1
content-type: application/x-www-form-urlencoded
X-Forwarded-Proto: https
and the response as received by the browser
content-length:0
date:Wed, 10 Jan 2018 15:45:21 GMT
expires:Thu, 01 Jan 1970 00:00:00 GMT
location:https://jenkins.default.mydomain.tld/
server:Jetty(9.4.z-SNAPSHOT)
set-cookie:JSESSIONID.d3f1b83b=node01o.node0;Path=/;Secure;HttpOnly
set-cookie:ACEGI_SECURITY_HASHED_REMEMBER_ME_COOKIE=Y2N1c2hpbmc6MTUxNjgwODcyMTQ5OTpmMmIzNDVkYjE2ODdhRlMGJmNWYz;Path=/;Expires=Wed, 24-Jan-2018 15:45:21 GMT;Max-Age=1209600;Secure;HttpOnly
status:302
via:1.1 6d16d2818.cloudfront.net (CloudFront)
x-amz-cf-id:WZMDdUGVK4fRg6E04gN2sr9RIgKDqeK0-kAANBbeu7eB3uFv9cAkQw==
x-cache:Miss from cloudfront
x-content-type-options:nosniff
I then see a subsequent GET /login HTTP/1.1 but I don't understand why it'd be doing that. Also possibly of interest, there is no default view set in configure globals, in fact the configuration for that seems to be absent. accessed over http, via haproxy everything is fine well the warning still shows, seems I can fix that by setting the jenkins uri to use http://IP:port.
how do I fix my haproxy setup such that jenkins login will not show the login form where the dashboard should be? can anyone explain the transformations that need to be done in a proxy technology agnostic way and why (.e.g. examples seem to show 127.0.0.1:8080, but htat's obviously not going to work for me, however since Location is returning https, I'm not sure why I'd need to do anything like that)?
I want my persistence (ebs) volume to be separate from my auto-scaling group launch configuration. So that it won't be accidentally auto deleted by terraform, or something. Also I may do something like /mnt/taskname so that I have a different persistence volume for each task that needs data.
Here's how I think it would work.
Problem is that this won't work though because 1 wouldn't happen until after 3 and 4 (I think), and I'm not really sure if 2 is the right way to go to attach the volume.
Maybe I shouldn't be using the ecs optimized image? seems like that would solve my issue, but then I'd have to figure out what to change the amazon (base?) image to have it otherwise configured like the ecs image? maybe there's another way to attach the volume than the aws command? maybe some other magic than what I'm considering entirely?
I'm trying to get ASP.net StoreFront to Connect to SQL Server Express 2005, on Windows Server 2008. I followed This Microsoft Support article without success. Below is the error.
[SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)]
AspDotNetStorefront.Global.Application_BeginRequest(Object sender, EventArgs e) +286
System.Web.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +80
System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +171
On a Linux System if I were experiencing this error I would use the SQL specific connection client to test the connection (e.g. mysql ). Possibly use telnet to test the port, follow up with iptables if that was the problem. I'm not sure how to go about doing the same things on windows. I've authenticated using SQL Server Management Studio Express using the same credentials? I put a question mark because I'm a bit unsure about one part of the datasource (that should be host/server identifier). I don't think telnet would test the port because windows firewall operates on L7 (it checks the application). How can I find out if the problem is my ability to connect to the server? or the configuration that defines how to connect to the server? (note: security is irrelevant to me, this VM is for development purposes only and is behind my laptops firewalls and has no NAT)
e.g. can I do
subdomain IN CNAME @
in a zone file? or is it only valid in the beginning of a record? e.g
@ IN TXT "v=spf1 a mx -all"
I was running postgres 8.4 on arch but now have 9 installed I want to use pg_upgrade to upgrade but I don't know what all the options I have to use are, and all the directories that I have to point it to. it asks for a previous pg binary directory... but that's gone. Will it still work? note: I've no important data, just play db's I just want to know how to do it.
I've got a nameserver that I updated a zone file on, incremented the serial, and ran rndc reload. I then verified that named saw it in the log, and it did, without errors. But when I run dig domain.tld @localhost note: @localhost is the bind server I made the changes on. bind still see's the old A record, in its cache. Do I misunderstand how bind works? or is this a problem?
I recently got a new job, and realize that I'm way to ignorant on mail troubleshooting. How do you recommend learning how to read email headers, mail logs, and troubleshoot various mail problems? any good books? tutorials? anything else? is the documentation comprehensive enough to start there?
Is it possible to filter (grep) exim_mainlog for emails that had attachments? I'm looking through a really long log for an email of which I don't have any information about other than the sender and that it had an attachment. Sender alone returned hundreds (POOMA) of entries. So I'd like to narrow it down.
I've been trying to get my raid to boot from grub2 for about 2 days now and I don't seem to be getting closer. The problem appears to be that it doesn't recognize my raid at all. It doesn't see (md0) etc. I'm not sure Why or how to change this. I'm using mdadm, 2 device (essentially a raid1) raid10,f2, which is currently degraded.
I have tried adding the raid and mdraid modules with grub install along with others. I've tried several variation on grub-install such as grub-install --debug --no-floppy --modules="biosdisk part_msdos chain raid mdraid ext2 linux search ata normal" /dev/md0
I've been searching the net for an answer to what I haven't done but no luck. On my other drive which I plan on removing the raid is initialized and mounted fine on boot, but it's not the boot/root for that setup. My grub.cfg isn't recognized by grub since it can't read the raid partition so I'm not posting that. md0 is not listed in my /boot/grub/device.map.
I'm building a raid1 (ok it will probably be a raid10,f2 but the difference with 2 drives... isn't much) system with 2 1TB drives. However, 1 of the drives I've ordered is bad so I'm RMA-ing it. I'm wondering if I could partition and install to the 1 drive and then rebuild the array when I get the second drive (after I test it of course) My initial investigation doesn't show me a way of creating the array without specifying all devices... and the device the second drive will be is one that has data that I will need to migrate (plus it's not big enough). Is it possible that I could create an array without specifying all devices? or specify false ones and reconfigure to the right ones later? Or some other method I'm not thinking of.
I been running smartctl -t long on a drive for about 2 days now and it seems to be stalled at 10%. short and conveyance both passed. I have to send 1 of 2 drives purchased back I found badblocks with badblocks (none on this drive and I'ts made over a pass already). I'm just wondering if I should be concerned about this.
smartctl 5.39.1 2010-01-28 r3054 [x86_64-unknown-linux-gnu] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net
=== START OF INFORMATION SECTION ===
Device Model: WDC WD10EARS-00Y5B1
Serial Number: WD-WMAV51582123
Firmware Version: 80.00A80
User Capacity: 1,000,204,886,016 bytes
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: 8
ATA Standard is: Exact ATA specification draft version not indicated
Local Time is: Mon May 10 22:19:52 2010 EDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 241) Self-test routine in progress...
10% of test remaining.
Total time to complete Offline
data collection: (20100) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 231) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SCT capabilities: (0x3031) SCT Status supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 2
3 Spin_Up_Time 0x0027 131 131 021 Pre-fail Always - 6408
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 12
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 148
10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 10
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 7
193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 174
194 Temperature_Celsius 0x0022 106 102 000 Old_age Always - 41
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Conveyance offline Completed without error 00% 99 -
# 2 Extended offline Interrupted (host reset) 10% 30 -
# 3 Short offline Completed without error 00% 0 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
I know how most of the various RAID's work. But I stumbled on the recommended raid10,f2 mode while researching linux software raid. I don't really understand how it works on 2 or 3 disks. could someone explain it to me? or point me to a really good article that explains it?