Multiple File Upload to Google Drive Using Google Script – Christopher Furton’s Personal Website
Multiple File Upload to Google Drive Using Google Script – Christopher Furton’s Personal Website
Written by: Christopher Furton
## Create CSV file for from TSP.Gov website for import into Quicken 2015
from lxml import html
# Function to retrieve data from TSP website
page = requests.get('https://www.tsp.gov/InvestmentFunds/FundPerformance/index.html')
tree = html.fromstring(page.content)
dates = tree.xpath('//td[@class="leadingCell"]/text()')
values = tree.xpath('//td[@class="packed"]/text()')
funds = tree.xpath('//th[@class="packed"]/text()')
# Function to parse data and then export to file
#Change dates from 'Apr 10, 2016' to '4/10/2016
while(x < len(dates)):
date = dates[x]
dates[x] = datetime.datetime.strptime(date,'%b %d, %Y').strftime('%m/%d/%y')
x += 1
#Change fund names to match Quicken Ticker Symbol
while(x < len(funds)):
fund = funds[x]
if fund == 'L Income':
elif fund == 'L 2020':
elif fund == 'L 2030':
elif fund == 'L 2040':
elif fund == 'L 2050':
elif fund == 'G Fund':
elif fund == 'F Fund':
elif fund == 'C Fund':
elif fund == 'S Fund':
elif fund == 'I Fund':
x += 1
#remove excess spaces and /n from share values
while(x < len(values)):
temp = values[x]
values[x] = temp[4:11]
x += 1
#Format into CSV rows and write to file
fo = open("exportToQuicken.txt", "w")
for date in dates:
while(colNum < len(funds)):
fo.write(funds[colNum] + "," + date + "," + values[itemNum] + "\n")
itemNum += 1
colNum += 1
dates, values, funds = retrieveData()
Christopher Furton is an Information Technology Professional with over 12 years in the industry. He attended The University of Michigan earning a B.S. in Computer Science and recently completed a M.S. in Information Management from Syracuse University. His career includes managing small to medium size IT infrastructures, service desks, and IT operations. Over the years, Christopher has specialized in Cyber Security while working within the Department of the Defense and the United States Marine Corps. His research topics include vulnerability management, cyber security governance, privacy, and cyber risk management. He holds active IT Certifications including the CISSP, CEH, ITIL Foundations, Security+CE and Network+CE. He can be found on LinkedIn, Google+, and Twitter @ChrstphrFrtn.
Written by: Christopher Furton
Photo Credit: Military.com
It seems as if nearly all companies proudly exclaim that they hire veterans – whether through official programs aimed to hire X number of Veterans per year, flocking to Hiring our Heroes career fairs, or through veteran affinity programs. Many people have likely read several “top 10” lists with resounding reasons to hire those who have served. That is great; those benefits are simply undeniable. We know veterans have adaptability honed through proven experience in unstable environments, leadership skills far beyond what would be typical for someone early in their careers, and technical proficiency only seen through years of education and hands-on application. However, that is not what this article is about. Instead, let’s dive further into the intent behind these programs and ask, “Why do we hire veterans and can we do more?”
Talent acquisition and recruiters likely dance with glee when that diamond candidate – the one with the perfect resume and interview skills – leaves military service and ventures out into Corporate America. They see those benefits and the potential positives that can be achieved by the company and rush to suck them into their fold. If that candidate has an active security clearance, then even better! No doubt this is good for the Veteran, but the intent seems rather selfish. It isn’t about helping that “perfect” veteran, it is about reaping the benefits. So let’s take a look deeper at what it means to be a veteran today.
According to the “Swords to Plowshares” report from the Institute for Veteran Policy (2011):
With the frightening increase in PTS, alcoholism, drug use, or other addictions, hiring a veteran may reveal that diamond candidate has the dreadful red flag – that item in a candidate’s past that is simply too much risk. This red flag could come in many forms: domestic violence or criminal convictions, employment gaps caused by residential drug programs, or possibly homelessness.
At this point, you may be asking yourself why I’m trying to convince you NOT to hire veterans! That of course is not my intent. Instead, I urge hiring managers and human resource professionals to ask yourselves if you are doing everything to help THOSE veterans: that percentage of veterans who need an extended hand the most. I like to refer to them as veterans with scars. Specifically, I’m referring to those invisible mental scars that may only be visible through unhealthy decisions and choices. At first it may look risky, but remember that some risks are worth taking. Veterans do come with many benefits, but more and more have lived complex and difficult lives as a result of service to our country.
Veterans are not perfect and I doubt that anyone claims they should be. With that said, there is so much opportunity that remains for companies to go the next step: selflessly hire veterans with scars. Let’s change the flight response when learning of a red flag and seriously consider the candidate based on his or her merits. You will likely get a dedicated and competent new employee while also extending a hand to someone who has sacrificed so much.
Written by: Christopher Furton
Originally Posted at: http://christopher.killerpenguin.net/blog/threatreport-ransomware
written by: Christopher Furton
Industry has experienced a 4,000 percent increase in crypto-ransomware attacks where generic ransomware grew at 113% in 2014 ( (Symantec Corporation, 2015, p. 7). Traditional ransomware attacks trick victims into paying a “fine” for accessing illegal or stolen content. This is typically done by a threat actor portraying to be a government official (i.e., FBI Agent) with official looking banners and websites (See figure 1). A victim can often escape this trap without paying any fees or fines. In contast, the crypto-ransomware attack holds a victim’s files and other digital media hostage by encrypting the contents and offering to sell the victim the decryption key. These ransoms can range from $300-$500 without any guarantee of successful decryption (Symantec Corporation, 2015, p. 7).
Windows environments are more typically affected by crypto-ransomware; however, Symantec reports seeing an increase in versions developed for other operating systems and mobile devices. Additionally, some crypto-ransomware is designed to attack network attached storage (NAS) devices and rack stations namely from Synology (McAfee Labs, 2015, p. 16).
A fairly new variant of crypto-ransomware named CTB-Locker is distributed through nested .zip files with a screen saver executable file. Transmission mediums include peer-to-peer networks, Internet Relay Chat, newsgroup postings, and email spam. Additional variants include CryptoWall, TorrentLocker, BandarChor, and Teslacrypt (McAfee Labs, 2015, p. 14).
Figure 1 – Sample ransomware attempt
Crypto-ransomware attacks increased dramatically up to 45 times more frequent in 2014 compared to the prior year (Symantec Corporation, 2015, p. 7). For organizations that run predominately Windows, this threat is in a higher risk category. The potential impact to the business of a successful crypto-ransomware attack is potentially devastating. Fortunately, the likelihood of a successful attack can be greatly reduced through mitigation techniques. Including ransomware into an organizations Enterprise Risk Management (ERM) program is advised as well as conducting a deep-dive into existing security controls to ensure proper mitigation efforts are in place.
Currently, there is no way to recover data encrypted in a crypto-ransomware attack. However, in some cases where law enforcement successfully shuts down a control server, recovery tools can be produced.
In Brief: Crypto-ransomware is often distributed through phishing attacks on users. According to McAfee Labs, at least one in every 10 is successful (p. 22).
AT-1 – Security Awareness and Training Policy and Procedures
This control outlines the higher governance for a Security Awareness Training program.
AT-2 – Security Awareness Training
This control outlines training for new users and periodic re-training.
CSC 9-1 – Build training and awareness roadmap
This control requires building a training awareness roadmap based off gap analysis of user behaviors.
CSC 9-2 – Deliver Training
This control required delivery of training by internal staff or external teachers.
CSC 9-3 – Online Security Awareness Program
This control outlines five steps to having a successful online awareness training program.
In Brief: Crypto-ransomware is ineffective if the organization can recover the data being held hostage with little impact to business productivity.
CP-9 – Information System Backup
This control outlines details of creating user-level, system-level, and security-related documentation back up.
CP-6 – Alternate Storage Site
This control establishes a geographically distinct alternate storage site including necessary agreements to permit the storage and retrieval of backup information.
CP-10 – Information System Recovery and Reconstitution
This control provides for recovery and reconstitution of the information system to a known state after a disruption, compromise, or failure.
CSC 8-1 – Data Recovery Capability – Backup
This control requires backup of data at least weekly but more often for sensitive information.
CSC 8-2 – Data Recovery Capability – Restoration
This control requires testing restoration capability of backed up data.
CSC 8-3 – Data Recovery Capability – Protection of Backup Media
This control requires proper protections of backup data commensurate with the sensitivity contained on the media.
CSC 8-4 – Data Recovery Capability – non-addressability
This control ensures that at least one backup destination is not continuously addressable through operating system calls. **Very important for crypto-ransomware mitigation activity
This information was compiled from the Unified Compliance Framework website at http://www.unifiedcompliance.com
Nearly all regulatory models require a level of data backup procedures including the following:
Written by: Christopher Furton
Original post here: http://christopher.killerpenguin.net/blog/information-architecture-techniques-and-best-practices
Information Architecture Techniques and Best Practices
Written by: Christopher Furton
The Internet of Things in the Retail Industry
The Internet of Things (IoT) is described as a paradigm where everything of value will communicate in a networked form with one another. This involves development of sensor network technologies that ultimately form a new standard in which information and communication is embedded within our environments. Objects, such as refrigerators and coffee cups, interact with other objects within a designated geographical range, all with the intention of improving the lifestyles of people (Yan, Zhang, Yang, & Ning, 2008, p. Ch. 13 Para. 1).
To accomplish this, three criteria are important: sensors and actuators, connectivity, and people and processes. First, sensors and actuators create a “digital nervous system” through the use of Global Positioning System (GPS) data, cameras, microphones, temperature sensors, pressure sensors, and many more. Next, connectivity moves that data across networks such as Personal Area Networks (PAN), Local Area Networks (LAN), or Wide Area Networks (WAN) using a variety of different protocols and architectures including cellular 3G/4G LTE, WiFi, Bluetooth, Near Field Communication (NFC), and many others. This connected data is combined into bi-directional systems where people and processes can utilize it to make better decisions, both human and automated (Harbor Research & Postscapes, 2015).
From a consumer products perspective, the Internet of Things offers potential for new product lines to turn everyday homes into “smart homes” by developing supporting architectural equipment as well as end-point devices like smart lightbulbs (Entertainment Closeup, 2015, p. 1). Over the past several years, smart consumer products have hit retail product portfolios putting home automation and the Internet of Things “safely poised on the brink of entry into the mainstream market” (Koyfman, 2014, p. 30).
Consumer products already on the market can adaptively adjust a home’s thermostat based off usage patterns and occupancy, continuously capture health information like calorie expenditure and temperature 24 hours a day, and remotely turn off appliances using a cell phone by terminating outlet power. However, this is just the beginning as Internet of Things devices are spreading throughout industries effecting home consumers, transport mobility, health care, building infrastructures, and industry practices (Harbor Research & Postscapes, 2015).
As the Internet of Things edges close to the mainstream in the consumer products market, it also has substantial potential value in backend operations. Technologies like Radio-Frequency Identification (RFID) allow objects to use radio waves to transfer information to readers without direct line of sight, potentially driving supply chain efficiency (Bardaki, Kourouthanassis, & Pramatari, 2012, p. 233). Through innovation, companies can leverage IoT technologies to outperform competitors and gain competitive advantage.
The Internet of Things technologies will likely affect all industries with transformational affect throughout retail, manufacturing, finance and insurance, and information services. According to Readdy (2014), retail has the second largest potential for gain from IoT technologies at US$1.6 trillion behind manufacturing at US$3.9 trillion. Information services and finance and insurance are tied for third and fourth with US$1.3 trillion of potential gain (p. 3). In total, estimates are upwards of US$14.4 trillion throughout all industries. These gains are anticipated through improvements in customer experience for US $3.7 trillion, innovation for US $3.0 trillion, supply chain and logistics for US$2.7 trillion, employee productivity for US$2.5 trillion, and asset utilization for US$2.5 trillion (Readdy, 2014, p. 3).
With retail poised to benefit upwards of US$1.6 trillion, early adoption offers potential to achieve competitive advantage. Particularly, retail companies can benefit from stock-out prevention, i.e. prevent empty shelves, because of connected and intelligent supply chains. Furthermore, the IoT opens the door for innovative use of technologies for predicting customer behavior and trends by performing big data analytics on data collected from video surveillance cameras, social media, Internet browsing, and mobile devices. Although the manufacturing industry is predicted to gain the most from IoT, retailers have the potential to build strong business cases for enhanced revenue, increased efficiencies, and improved asset management (Readdy, 2014, p. 2).
Strategy Implications for Retailers
The Internet of Thing will affect strategy through six key areas: energy, security, smarter analytics, new revenue streams, productivity, and travel. The first strategic area, energy, is similar to the concepts discussed in the Overview regarding consumer products. Through the use of connected devices that regulate lighting and temperature, retail stores as well as distribution centers and backend offices will reduce energy expenditure (Anderle, 2015). This “greening” of facilities utilizes Information Technology strategy in the form of the IoT to intelligently reduce energy expenditure without compromising productivity or business functionality.
The second strategic area, security, will primarily affect retail organization’s Loss Prevention (LP) or Asset Protection (AP) divisions. Because these LP/AP divisions are responsible for physically securing property with locking mechanisms, closed-circuit television (CCTV), and employee and customer safety, the IoT will have significant impact. These functions exist at store level, distribution level, and corporate level alike. Smart locks will keep track of who is in the buildings at any particular time, smart doorbells will inform employees of who is trying to gain access, and smart surveillance systems will potentially save substantial man-hours reviewing video footage for specific events (Target Corporation, n.d.) (Harbor Research & Postscapes, 2015).
The third strategic area, smarter analytics, is one of the most important areas that offers significant benefits to retailers through IoT. Essentially, more smart devices means more data collection for analytics that, with the right people and processes, can improve strategy and customer experience (Anderle, 2015). Historically, e-commerce sites have been able to leverage analytics to figure out where items should be positioned on web sites to catch the eyes of shoppers and suggest follow-up purchases. With the IoT, retail stores will be able to track customer movement throughout the store, analyze pauses in movement and collect data points for analysis. RetailNext, a comprehensive in-store analytics company, already offers a service that provides 10,000 data points per store visitor. This video-based service collects 57 petabytes per year from 300 million shoppers at 50 retail chains (Groenfeldt, 2012).
In addition to smarter tracking of customers through stores for marketing purposes, the IoT analytics enables multi-channel functionality in brick and mortar stores. Through cross channel integration of back-end systems of product information, inventory, promotions, Customer Relationship Management (CRM) along with smarter physical shopping aisles, retail companies can achieve in-aisle consumer interaction through the use of mobile devices (Wonnagy, 2011). For example, based off Internet browsing, a retailer learns a consumer is interested in purchasing a new coffee brewer. The CRM system keeps track of this data and provides it to the physical store when the customer’s cell phone passes a sensor in the entry doors. Based off product information, inventory, and promotions, a push notification is sent to the consumer’s device with an advertisement and directions to the appropriate aisle number.
The fourth strategic area, new revenue streams, moves the IoT beyond cost reduction into a profit center driving sales and profitability. First, retailers can profit from added product lines directly related to IoT technology consumer products. According to forecasts, the sale of connected devices and related services could result in US$2.5 trillion in revenues by 2020. Forecasts also predict the number of smart devices to exceed 50 billion and machine-to-machine connection, the fundamental backbone of the IoT, to grow to 18 million, up from two billion in 2011. This growth, combined with the declining sensor cost, increase in computing and processing power, low-cost data storage, and widespread high-bandwidth connectivity, positions the retail companies to exploit substantial revenue growth by adding IoT consumer product lines to existing portfolios (Readdy, 2014, pp. 3-4).
The fifth strategic area, productivity, offers the benefit of increased efficiency to reduce costs. Currently, two thirds of organizations that have IoT solutions report having achieved 28 per cent cost reduction in daily operations (Inside Retail, 2015). Specifically, retailers can benefit through better supply chain management, inventory, logistics, and fleet management. Currently, bar code and RFID technologies let retailers monitor inventory levels, but IoT technologies will increase the data coming in to these monitoring systems. This provides better insight to products moving through the supply chain leading to improved efficiencies and leaner inventories (Sankaran, 2014, p. 1).
According to research by The Economist Intelligence Unit (2015), companies see improving productivity as the key immediate benefit to IoT despite long-term expectations for revenue growth (p. 5). Productivity benefits include improved overall employee production, optimized utilization of assets, reduced operational expenses, improved Internet oversight and control, and enhanced worker safety (The Economist Intelligence Unit, 2015, pp. 8-9).
The sixth strategic area, travel, affects retail logistical fleets as well as corporate travel via commercial airlines. Updating retail logistical fleets with smart technologies can increase safety and potentially avoid costly accidents. IoT technology may allow trucks to interact with other vehicles on the roadways through predictive algorithms and models providing best escape avenues for drivers in emergency situations or to identify potentially dangerous drivers. Current technologies include self-braking vehicles but it is anticipated that the IoT will develop with vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) systems. For example, Ford’s safe car technology converts intersections into “smart intersections” that can predict when a driver is going to run a red light and transmit warning to surrounding vehicles (Bertolucci, 2013, p. 1).
In addition to upgrading retail fleets, IoT has strategic benefits for retail companies through increased efficiency in commercial airline travel. Despite the growth of virtual collaboration, travel is a necessary evil of business that will not go away. Fortunately, the IoT has potential to improve the airline industry through better flight planning and operational changes that will have secondary effects on retail executive travel (Readdy, 2014, pp. 2,4).
Potential Threats and Challenges
Although the Internet of Things can be exploited to improve revenue streams, reduce costs, and enable innovation leading to competitive advantage, there are four challenges that must be considered. First, lack of standards and industry-wide agreement on protocols may introduce interoperability problems down the road. Second, cybersecurity concerns over network-enabled sensors and devices must be considered to prevent misuse or abuse.
Next, existing infrastructure must be able to support the increased bandwidth resource utilization caused by IoT devices. Lastly, retail culture in respect to employees and customers must support the increased data gathering and potential privacy concerns arising from Big Data initiatives including IoT.
First, with so many stakeholders in the IoT, achieving widespread use of standards will take some time. Likely protocol wars will emerge especially as legacy IoT device companies try to protect their proprietary systems. Open system proponents will likely push for industry standards to encourage better systems integration (Kocher, 2014, p. 1). Having universal standards would also reduce risk regarding security concerns.
Second, security concerns must be considered. The increase in devices leads to more decentralized entry points for malware. Devices will likely be placed in physically accessible areas which could be subject to tampering and exploitation. The increase in software, middleware, programmer interfaces, and machine-to-machine communications results in additional complexity and security requirements. These can be addressed with internal policies, but likely commercial products that leverage a policy-driven approach and provisioning will become available (Kocher, 2014, p. 1).
Weak information and telecommunication infrastructure is another concern when considering IoT initiatives. According to a report from the Economist Intelligence Unit (2015), 44% of companies surveyed identified poor information and telecommunication infrastructure as the one of the most significant obstacles to developing the IoT (p. 9). As the number of devices increases, there may also be a shift in the use patterns of network bandwidth demand. Estimates show an increase in global peak traffic (per hour) from 2,823Gb in 2012 to 16,215Gb in 2020 (Zhuang, Cappos, Rappaport, & McGeer, 2013, Table 3). The increased processing power to perform analytics, storage space to maintain databases, and communication pathways supporting wireless solutions, cloud, and mobile computing must be in place to support an IoT initiative (Zhuang, Cappos, Rappaport, & McGeer, 2013, pp. 10-11).
Lastly, privacy concerns must be considered and cultural changes made that support the collection and analysis of significant amounts of consumer information. Retail customers may object to being tracked throughout the retail brick and mortar stores despite similar current practices in e-commerce. Trust must be developed between consumer and retailer so that information can be gathered without fear of abuse or compromise (Kocher, 2014, p. 1).
Exploiting Opportunity – The Path Forward
Despite the challenges presented by the Internet of Things, retail companies are uniquely positioned to develop business strategies that leverage this cutting-edge technology. Making the move towards an IoT ecosystem requires careful consideration of the fit within business, organizational, and information systems strategy. As described in Pearlson & Sanders (2013), the company will need ensure the organizational strategy is in alignment with the business and information systems strategies creating the Information Systems Strategy Triangle (pp. 23-24).
First, business strategy needs to be assessed to determine which specific areas an IoT technology can help achieve desired results. For example, as a retailer, there are several strategic functions where IoT can be beneficial — namely, supply chain management, marketing (analytics), multi-channel sales, and improved in-store checkout process. Despite the temptation to integrate IoT into all these functions simultaneously, a “rip and replace” mentality is not advised. Because retailers will not be able to create use cases from scratch, they will need to integrate the new technologies with current systems, data, and infrastructure investments. This is best done by ensuring alignment of IoT with existing company strategies. Furthermore, the company needs to avoid treating IoT as a technology experiment by building a technical solution in search of a problem. The IoT needs to meet business objectives and be directly linked to business strategy (Kocher, 2014, p. 2).
One method of doing this would be to develop a phased deployment of IoT technologies focusing first on small scale opportunities that enable quick wins without jeopardizing existing processes. For example, developing a marketing/analytics strategy where IoT sensors placed strategically in pilot stores increase customer touch-points to improve available analytic information on customer behavior. The same IoT architecture could be further expanded over time to include multi-channel sales. The next big step would be phasing out barcodes replacing with RFID leading to better supply chain management options with an end-state goal of eliminating slow moving checkout lines where cashiers scan barcodes (Bardaki, Kourouthanassis, & Pramatari, 2012).
This is a big feat that requires modifications to Information System strategy so that alignment is achieved with business strategy. The IoT ecosystem will need to encompass existing data structures and create many new ones. For retail organizations, that means linking loyalty card programs with Internet browsing activity and linking brick and mortar customer touch-points into these data warehouse repositories. Also, employing a “Green IS” into the phased IoT implementation can help reduce costly energy consumption.
Lastly, to achieve alignment under the Information Systems Strategy Triangle model (Pearlson & Saunders, 2013, pp. 23-24), the retail company will need to consider evaluating organizational hierarchy and assessing manpower requirements to achieve such a large scale innovation initiative such as IoT. People and processes are critical for an IoT initiative which places great emphasis on the need for organizational strategy alignment. An increase in qualified Information Technology professionals, as well as experienced marketing analysts and logisticians, will be required to properly fulfill the IS strategy with the end goal of successful business strategy and competitive advantage.
The Internet of Things will have a profound impact on businesses and consumers in the near future. Most industries will develop innovative ways to leverage the power of the IoT through the use of sensors and actuators, extensive connectivity, and intelligent people and processes. In particular, companies within the retail industry are positioned to gain significant benefits for early adoption impacting several strategic areas: energy, security, smarter analytics, new revenue streams, productivity, and travel. More specifically, creative use of IoT can enable retailers to increase revenue through additional product lines, decrease costs through lean processes and green initiatives, and enable technical innovation through next-generation supply chains and cashier-less RFID checkouts.
Despite these opportunities, the IoT does have several challenges that must be considered. Lack of industry standards, weak cybersecurity posture, inefficient or nonexistent infrastructure, and privacy concerns must all be addressed when planning an IoT initiative. Regardless, the rewards are greater than the risks so the IoT opportunity should still be explored. Through business, organizational, and information systems strategy, a phased implementation allowing opportunity to adjust to consumer demands and build trust, expand existing infrastructures, and ensure security measures are sufficient will position the retail company favorably to profit in the short and long-term.
Topic Area 2: Overview of Exhibit 300s
Originally posted here: Mitigating Botnet Information Security Risks Through EA and the ITSA
Part 1 of 4
Often referred to as zombies, malware compromised computers take part in criminal cyber activity without the knowledge of their owners. Zombies are members of large networks called Botnets. These networks range in size and complexity, but all have serious implications to enterprise security. As a tool for criminal activity, botnets can ‘earn’ criminals substantial revenue by engaging in spam mass emailing and information theft campaigns. Similarly, some criminals generate revenue by renting access to their botnets to other cyber criminals (Ferguson, The history of the botnet – Part II, 2010). Besides financial gain, botnets are a common tool for hacktivism where hackers use malicious attacks to further a political viewpoint (Schectman, 2012).
This paper explores the world of botnets. The paper is broken into four parts: 1) The Problem; 2) The Mitigation; 3) The Case Study; and 4) The Conclusion. In ‘Part 1- The Problem’, the goal is to explain the types of technologies utilized in botnets and identify the potential risks associated with them. In ‘Part II – The Mitigation’, the goal is to offer recommendations for combating botnet risks specifically through the use of proven methodologies such as the Bernard & Ho’s Information Technology Security Architecture (2008). In Part 3 – The Case Study, a real life look at a nation state that used business continuity planning to reduce the impact of a botnet distributed denial of service attack. In ‘Part 4 – The Conclusion’, the goal is to tie the main points of this paper together.
This section of the paper provides background information on botnets and identifies the problems faced by internet users and the enterprise environment. The contents include: a brief technical overview, explanation of propagation techniques, topology differentiation for command and control, discussion on intended targets, typical use of botnets, and the history of botnets. The aim of Part I is to ensure an understanding of botnets and introduce the problems that they cause to the enterprise environment.
In order to discuss propagation techniques, it is first important to clarify that botnets are a network of compromised hosts. Developing a botnet occurs by infecting vulnerable computers with command and control malware giving the botmaster control of the newly created bot. When discussing propagation techniques, this paper focuses on activities used by botmasters to initially infect vulnerable computers.
In the beginning stages of propagation, botnets look for vulnerable hosts that have unpatched operating systems or software applications. The methods used to exploit these vulnerabilities are often controlled by the botmaster during propagation. Successful botnet propagation relies on a controlled rate of infection that doesn’t interfere with network stability. Too rapid of propagation can result in network instability and reduce the overall effectiveness of the botnet (Xin-liang, Lu-Ying, Fang, & Zhen-ming, 2010).
In contrast to the preferred controlled propagation, some botnets spread similar to malware worms. In these instances, an already compromised host finds other vulnerable hosts and exploits them without influence from the botmaster. This form of propagation is wild and uncontrollable.
The propagation methods discussed above do not require user interaction. However, many botnets propagate in a matter that requires a user to perform a task. The first and most common method (Dagon, 2005) of propagation is by email. As seen in mid-2011, the ZeuS botnet used email to spread in the form of a fake IRS spam email. In this example, the emails appear to originate from the irs.gov domain where the subject reads “Your IRS payment rejected” or “Federal Tax payment rejected.” The body of the email refers the victim to an attached PDF file containing the ZeuS malware (MXPolice, 2011). Using social engineering tactics (the fear of IRS audit), the ZeuS botnet leveraged email as a method for propagation.
Another propagation method is through instant messaging. In this method, botmasters attempt various forms of attack through instant messaging including social engineering attacks attempting to lure the victim into clicking a malicious link. Additionally, the botmaster can send a malicious file to the victim and entice him/her into opening it (Dagon, 2005). As seen in the Mariposa botnet, which was shutdown in March of 2010, the instant messaging software MSN Messenger was used by threat actors to spread malicious code to unsuspecting victims (Kolakowski, 2010).
Web pages are also often used to spread malicious code that enables botmasters to increase the size of their botnets. In this method, webpages host content that installs malicious code on visitors computers permitting botmasters to gain control. As identified by WebSense (2008),
These figures show a potential change in threat climate pointing to internet web browsing as being a significant contributor to botnet propagation.
Lastly, botnets can exploit vulnerabilities in other malware already running on the host. For example, the Bagel and MyDoom worms contained backdoors that were exploited by botnets in April of 2004 (Cooke, Jahanian, & McPherson, 2005).
The first topology seen within botnets relied heavily on Internet Relay Chat for command and control. As the birthplace of botnets, IRC channels were used for running games, file distribution, and for user misbehavior. “Early bots were not always malicious” (Bu, Bueno, Kashyap, & Wosotowsky, 2010). In IRC botnets, the IRC channel acted as the command and control server for the compromised zombies. IRC traffic typically occurred over a particular port number from zombie client to IRC server (Bailey, Cooke, Jahanian, Xu, & Karir, 2009).
The next topology seen within botnets relies on peer-to-peer (P2P) communication for command and control. Instead of using a centralized architecture as seen in IRC botnets, P2P botnets allowed peers to connect to other peers as long as their IP address is known within the botnet database. The botmaster can inject commands to any peer within the botnet and the command is then relayed to other peers (Bailey, Cooke, Jahanian, Xu, & Karir, 2009). This type of botnet has many variations and has evolved to keep up with security researcher’s attempts to track down known peers. “In the last several years, botnets such as Slapper, Sinit, Phatbot, and Nugache have implemented different kinds of P2P control architectures” (Wang, Sparks, & Zou, 2010). Some have implemented cryptography for update identification and encrypted or obfuscated control channels. Although the botmasters have evolved the malware to defeat inherent weaknesses in P2P botnets, these modifications often open up new methods for detecting and compromising the botnet’s anonymity (Wang, Sparks, & Zou, 2010).
In this topology, botnets use standard web requests that operate over port 80 to facilitate command and control. This topology uses a webserver as the centralized command and control channel similar to how IRC botnets used IRC channels. However, the web server C&C channel stays always connected with eliminates the fundamental problem of connection loss to IRC channels. In HTTP botnets, the traffic flows with regular web browsing traffic. However, the HTTP botnet traffic is structured different than normal traffic making it easier to detect (Bailey, Cooke, Jahanian, Xu, & Karir, 2009).
One of the most popular HTTP botnets found in the wild today is the ZeuS botnet. ZeuS consists of both a client and a server component where anyone with little computer expertise can create a custom version of the malware. Ironically, the current version of ZeuS uses a strict commercial software license which links directly to the buyer’s physical hardware. “The creation and distribution channel of this malware displays a strong business sensibility” (Bu, Bueno, Kashyap, & Wosotowsky, 2010).
The last topology discussed is the newest growing for botnets. These botnets leverage Web 2.0 technologies often seen within social networking websites. Similar to HTTP botnets, Web 2.0 botnets utilize web applications such as Facebook, MySpace, RSS, and Blogging for command and control purposes. Although the concept of social network C&C dates back in academic work as early as 2007, the first reported botnet – named Naz – was found on Twitter.com and Jaiku.com (Kartaltepe, Morales, Xu, & Sandhu, 2010). The Naz command and control attack flow and control flow is diagrammed in figure 1 below. This type of botnet exhibits the increased complexity and innovativeness of botmasters.
In research conducted by Damballa (Ollmann, 2009), a distinguishing factor identified directly relates to what type of victim is targeted by a botnet: broad-spectrum internet user or the enterprise asset. In this research, 50 percent of botnets identified in the enterprise environment were Internet Targeted botnets. These broad-spectrum attacks are aimed at any Internet user but often enter enterprise environments due to relaxed security or usage of personally owned computing equipment in the workplace. These botnets often have readily available fixes but require enterprise security teams to patch software properly and keep anti-virus signatures up to date (Ollmann, 2009).
The next target group identified is called the Enterprise Targeted botnets. In this case, botnets found within the enterprise are hardly ever found circulating the Internet. These botnets are designed to penetrate and propagate within enterprise networks and are a blend of sophisticated remote access Trojans with worm propagation functions. These botnets are often targeted at specific industries such as online retail companies or specific people within the organization such as the Chief Financial Officer. These botnets are typically more advanced than Internet Targeted botnets. Around 35 percent of botnets encountered within the enterprise are of this type (Ollmann, 2009).
The next group identified is called the Deep Knowledge botnet. Although only making up 10 percent of the botnets identified in the enterprise, these botnets can be very sophisticated and very dangerous. The botmaster often has a high degree of knowledge about the infiltrated enterprise and the information architecture. It is believed that many of the Deep Knowledge botnets are created and installed by hand for legitimate remote administration by employees. The bigger problem is that many commercial do-it-yourself malware construction kits have backdoors to their creators or partners (Ollmann, 2009).
That last group identified by Damballa is a catch-all group referred to as Others. In this group, the remaining 5 percent of botnets encountered in the enterprise vary in sophistication and functionality and don’t fit neatly into any other group. These include small botnets targeted at a specific group for industrial espionage and competitive advantage or possibly state-sponsored botnets aimed at specific goals (Ollmann, 2009).
Because of the flexible nature of botnets, the use by cyber criminals is vast and evolving. One common use of botnets is the execution of Distribute Denial of Service (DDoS) attacks. In a DDoS, botnets are used to deplete the network bandwidth and other computational resources of target sits. Using a botnet for this type of attack magnifies the impact of the attack and eliminates the need to mask or spoofidentifying information (Choo, 2007). In the enterprise environment, botnet DDoS attacks may pose a substantial risk particularly for e-commerce lines of business. Also, DDoS attacks aimed at unique network resources such as the Dynamic Name Service (DNS) may prevent normal business operations within the enterprise environment. Similarly, ‘spidering’ attacks on a company’s website uses HTTP floods that recursively access resources causing denial of service conditions (Uses of botnets, 2008)
In addition to DDoS attacks, botnets are also used for spam dissemination. In April of 2005, Symantec spam statistical report indicated that 61 percent of global email was identified as spam (Choo, 2007). The financial gain achieved by botmasters through spamming encourages ever increasing vigilance. A spambot malware, known as SpamThru, included sophisticated features that used advanced encryption, installs its own antivirus scanner to eliminate competing malware, and even enacted functions to evade anti-spam measures (Choo, 2007). Enterprises inflicted with botnet malware may be producing spam inside the enterprise.
Information theft is a major concern for botnets in the enterprise environment as well as individual privacy for home users. Sniffing traffic and key-logging components are often found in botnet malware allowing botmasters to collect unencrypted traffic passing through the bot or log all keystrokes entered by a user (Uses of botnets, 2008). In the enterprise environment, there is a substantial risk of compromising critical sensitive information or business trade secrets. This information must then by ex-filtrated back to botmasters through covert channels.
Botnets have also been used to spread new malware. Newly created malware can obtain a substantial rapid existence by using computers under the control of a botmaster to launch the new malware. Many botnets include functionality to remotely download new files and execute them. The Witty worm was initially launched through the use of an existing botnet (Uses of botnets, 2008). Botnets existing in an enterprise environment pose a substantial risk as newly released malware may not have antivirus signatures available magnifying potential compromise.
Another substantial motivator for botnet use is for financial gain. Often referred to as “click fraud”, botnets are able to abuse ad programs like Google AdSense by using bots to ‘click’ on ads to artificially increase the click counter. The use of this type of financial gain is not common (Uses of botnets, 2008); however, a 2010 study indicated a growth in this activity with 42.6 percent of all click fraud originating from botnets (Singer, 2010). A similar type of financial gain was seen with a recent Twitter-based botnet that mines the online currency known as bitcoins. This type of botnet was aimed at stealing virtual currency by leveraging the massive distributed computing power of the botnet to solve complex mathematical tasks. Based off the bitcoin economy, the more computations a user accomplishes the more virtual currency can be created. That virtual currency has exchange rates for conventional currency (The H Security, 2011).
Of greater concern than bitcoin mining, botnets can be used for mass identity theft. Botnets can deploy phishing scams that lure victims into entering sensitive private information into compromised or bogus websites like PayPal or banking institutions (Uses of botnets, 2008). This tactic combined with packet sniffing and key logging introduces substantial risk to the enterprise and the organization’s employees.
The origins of botnets can be traced as far back as 1999 with the creation of the malware Sub7 and Pretty Park. Both of these offered a control method utilizing an IRC channel where the creator could send malicious commands to infected computers. A year later, the Global Threat bot, or GBOT for short, was introduced that included higher sophistication. Namely, the GBOT was able to access raw network level sockets (both connection-oriented TCP and connection-less UDP) allowing for Denial of Service attacks. Additionally, the GBOT had the ability to hijack Sub7 infected computers and “update” them to GTBots (Ferguson, The history of the botnet – Part I, 2010).
In 2002, the release of SDBot and Agobot fueled the growth of botnets and initiated the creation of variants. These two botnets introduced techniques such as creating backdoors, disabling anti-virus, and blocking access to security vendor websites. These early botnets were aimed at information theft and remote control. SDBot, due to the public release of its source code, became the standard for several variants including the Spybot botnet in 2003. With Spybot came new functionality such as key logging, data mining, and Instant Messaging Spam (SPIM) (Ferguson, The history of the botnet – Part I, 2010).
Also in 2003, two more significant functionalities were first seen in the wild. First, the Rbot botnet introduced proxying for relaying commands and the coordinated Distributed Denial of Service (DDoS) attack. Rbot also included information stealing tools as well as encryption techniques to try to evade detection. Second, the Sinit botnet introduced a new topology of peer-to-peer. This marked the evolution of botnets away from the IRC command and control channels due to easy detection and frequent blocking at enterprise boundary firewalls (Ferguson, The history of the botnet – Part I, 2010).
Criminal interests surfaced in 2003 with several botnets that facilitated spamming. The Beagle, Bobax, and Mytob botnets included mass-mailing functionalities enabling criminals to distribute their spam with agility, flexibility, and covertly to avoid ever increasing law enforcement efforts (Ferguson, The history of the botnet – Part II, 2010).
Throughout the next several years, many famous botnets were introduced. RuStock in 2006 and the infamous ZeuS crimeware family. As an information stealing tool, ZeuS has been updated to newer versions several times with increased functionality and lethality. The botnet interfaces have been designed to entice less technically savvy criminals by allowing for simple point and click controls. Subsequently, developers have included backdoors in the command and control software turning criminal controllers of botnets into victims as well (Ferguson, The history of the botnet – Part II, 2010).
Efforts to fight back have been launched by government and private companies. In 2008, two Internet Service Providers de-peered – or stopped routing traffic – the McColo hosting provider which routinely hosted command and control servers for botnets. This takedown resulted in a 75% reduction in spam Internet-wide (Security Focus, 2008). In June of 2009, the Federal Trade Commission closed down the Internet Service Provider ‘3FN’ which impacted some botnet command and control networks. Despite efforts to disrupt these botnets, the creators become more innovate and increase efforts at evading detection. One technique used by the Conflicker botnet was to generate 50,000 alternative hostnames daily making in near impossible for the security industry to block them all (Ferguson, The history of the botnet – Part II, 2010).
In the late 2007s, the landscape of botnets continued to evolve into the Web 2.0 technologies. Having left behind IRC and basic peer-to-peer command and control, alternate channels were embedded in blogs and Real Simple Syndication (RSS) feeds. Criminal innovation continues to evolve as seen by ZeuS bot storing configuration files in the compromised Amazon EC2 cloud service. With botmasters using Facebook, Twitter, and Google as command and control channels, detection has become more and more difficult as communication to these sites is very common and expected. Finding the hidden, covert channels is and will continue to be challenge for security specialists. Future expectations include use of highly effective encryption techniques such as Public Key Infrastructure (PKI) and advanced peer-to-peer cloud services. Already in use, the Koobface botnet uses social networking services for propagation of spam by sending messages, making posts, and even creating it’s own Facebook profile page (Ferguson, The history of the botnet – Part III, 2010) (Ferguson, 2010 – Year of the Zombie Cloud?, 2010).