In a previous post, I noticed that all my Kubernetes services with type=LoadBalancer were exposing some internal services as NodePorts which meant that I might be exposing internal services to the Internet at high ports. I was running Kubernetes directly on my dedicated servers and not behind a load balancer. Kubernetes expected everybody to sit behind a LB which often times required a NodePort.
The solution was to set the Service spec.allocateLoadBalancerNodePorts value to false when the service is created. This works if I can set it while I create the Service, however Helm based templates often wouldn’t allow me to set this and once it was set to true and the node port was allocated it was difficult to deallocate the NodePort.
In this post, I walk through using a Kubernetes mutating webhook to automatically set the value for all Services.
I’ve worked on several different teams over the past 8 years I’ve worked at Amazon. Each one of them had on-call in which the engineers were on-call to keep the system running 24/7 for a week. If something broke at 2am, they’d get paged to fix it.
Now, Amazon’s a big company. On-call varied quite a bit. Some teams had more ops load, others had barely any. I had my fair share of weeks with lots of tickets, but usually I sought out teams where it was more manageable.
I was responsible for building a domain-specific authorization system that 20+ different services leveraged to secure the AWS hardware and Amazon Devices supply chain
Senior Software Engineer
Amazon.com, Inc.
Physical Stores Tech
Upon joining, I rapidly identified fundamental problems in the system and came up with a multi-phased plan to enable the Amazon Fresh store launch to unblock and launch (this was visible at the VP level) and longer-term projects to improve scaling and stability.
I set a long-term technical vision for a project spanning two teams and oversaw tech designs from other engineers to fit within that vision.
I worked with the product team to develop a product vision for a new internal template development editor that is expected to have a separate development team and product team.
Senior Software Engineer
Amazon.com, Inc.
Normalization Learning Systems
I worked to improve team productivity and robustness of changes by introducing design reviews/templates and improve team code quality output with standards.
I analyzed an existing team's processes and technical bar to identify weaknesses in team culture and engineering excellence and came up with multiple short-term and long-term plans to reduce technical debt.
Senior Software Engineer
Amazon.com, Inc.
Softlines Selection Systems
I improved developer productivity by recognizing gaps in Amazon’s existing alarming and service management and designed solutions using AWS X-Ray and open-source exception tracking tools. I am actively working to drive adoption from other teams and departments.
Software Engineer II
Amazon.com, Inc.
Softlines Selection Systems
I led high-impact projects with VP-level visibility from the requirements phase through to production. These projects required coordinating changes across multiple engineers and different teams requiring leadership and influencing skills.
I developed new business goals by identifying opportunities to improve productivity and increase revenue. I wrote business documents that were reviewed by senior stakeholders and developed these into technical project designs.
I consistently drove higher standards in code quality, architecture designs, and feature UX across multiple teams through design reviews, quality improvement campaigns, and code reviews.
Software Development Engineer
Amazon.com, Inc.
Softlines Selection Systems
I joined a brand new team that enabled Amazon to effectively plan selection of clothing/fashion products. I owned and drove code quality/infrastructure standards across the entire system (front-end three-tier and a back-end Apache Spark pipeline) using code analysis and code reviews.
I oversaw the design and implementation of an interactive OLAP-style analytics/deep-dive UI and service that implemented a custom query execution engine to interact with Elasticsearch. Queries were optimized to aggregate and search across billions of products in the Amazon catalog.
I directly worked with stakeholders to understand feature requirements and translated those into actionable technical implementation projects.
I contributed multiple changes to core Amazon frameworks, including a service communication framework widely used by teams across Amazon.
Software Development Engineer Intern
Amazon.com, Inc.
Forecasting Platform Orchestration
Developed custom algorithms to manipulate and optimize large-scale applications in a domain-specific language
Managed software platforms and frameworks used by Amazon to predict product demand for all physical assets across the world
Lab Manager
Northern Michigan University
Intel Lab
Developed hardware validation software for Android platforms to ensure expected functionality in the wireless subsystems (Bluetooth, Wi-Fi, LTE, etc)
Coordinated projects and development while working a semi-autonomous lab environment
Software Development Engineer Intern
Amazon.com, Inc.
Inventory Planning and Control
Designed and developed a new web service API used to interact with an existing data processing system
Helped setup and maintain a globally distributed and scalable data processing application by managing infrastructure capacity and minimizing outages
Improved the performance and scalability of an application by reducing bottlenecks and increasing maximum system utilization
Validation Engineer Intern
Intel Corporation
Wireless Platforms R&D
Developed automated test plans used to validate correct operation and find bugs for new wireless hardware devices
Developed several new applications and systems used by various teams to test the efficiency and performance of upcoming and release wireless platforms
Developed an in-depth test framework for the Android platform used to verify low-level functionality of Bluetooth, Wi-Fi, and cellular radio systems in upcoming mobile products
Developed a tool used by manufacturing technicians to quickly validate wireless cards from the factory assembly line
Software Development Engineer Intern
Amazon.com, Inc.
Vendor Ordering Systems
Improved and maintained a multi-tiered service written in Java and C# used by the purchasing-teams at Amazon to coordinate and optimize purchasing workflows
Designed and fully implemented a custom network communication protocol using WCF greatly increasing network efficiency and improving client-side user interface responsiveness
Lab Manager
Northern Michigan University
Intel Lab
Designed and developed an autonomous robot to be used as wireless device testing platform
Setup and maintained a small Ethernet network used for test device control using appropriate DHCP, Active Directory, etc. servers
Assisted and trained co-workers with new software and technologies used for their position
Managed the direction of a software development lab and all employees working in the lab
Network Technician
Northern Michigan University
IT Technical Services
Installed communication infrastructure serving computers, phones, printers, and other network devices
Installed and repaired devices such as switches and access points
Troubleshot and repaired computers in a network environment
Provided technical support for network-related problems from students, faculty and staff located on the University campus
Developed and maintained software used for testing the University's WiMAX network and mapping possible signal strength and performance problem spots
Inventory Manager/Software Developer
The Dutch Shop LLC
Developed and maintained several web projects that integrated third-party systems with a web-commerce web site in a small-business environment
Assisted in the general day-to-day operations in the business by managing inventory levels, communicating with customers, and updating the online product catalog