Home Tags Data Mining

Tag: Data Mining

Of the many use cases Python covers, data analytics has become perhaps the biggest and most significant.

The Python ecosystem is loaded with libraries, tools, and applications that make the work of scientific computing and data analysis fast and con...
Tabs and spaces aren't the only things that influence developer pay.
Organizations today are gathering ever-growing volumes of information from all kinds of sources, including websites, enterprise applications, social media, mobile devices, and increasingly the internet of things (IoT).The big question is: How can you derive real business value from this information? Thatrsquo;s where data mining can contribute in a big way.

Data mining is the automated process of sorting through huge data sets to identify trends and patterns and establish relationships, to solve business problems or generate new opportunities through the analysis of the data.[ Roundup: TensorFlow, Spark MLlib, Scikit-learn, MXNet, Microsoft Cognitive Toolkit, and Caffe machine learning and deep learning frameworks. | Cut to the key news and issues in cutting-edge enterprise technology with the InfoWorld Daily newsletter. ]Itrsquo;s not just a matter of looking at data to see what has happened in the past to be able to act intelligently in the present.

Data mining tools and techniques let you predict whatrsquo;s going to happen in the future and act accordingly to take advantage of coming trends.To read this article in full or to leave a comment, please click here
Twenty years of orbiting around a black hole ends up in perfect agreement with relativity.
Law professor Ahmed Ghappour tells us about changing rights at the US border.
Maltego, the tool best known for deep data mining and link analysis, has helped law enforcement, intelligence agencies and others in security-related work since it was released in 2008.

To benefit from using Maltego, come to SAS 2017 for intensive Digital Intelligence Gathering training from the experts who created the tool from scratch: there won’t be any questions that they can’t answer.
Enlarge / Mississippi Attorney General Jim Hood (R) at a news conference in 2015.Alex Wong/Getty Images reader comments 34 Share this story Mississippi Attorney General Jim Hood is sparring with Google once more. Last year, Hood and Google wound down a court dispute over Hood's investigation into how Google handles certain kinds of online content, from illegal drug ads to pirated movies.

E-mails from the 2014 Sony hack showed that Hood's investigation was spurred on, in part, by lobbyists from the Motion Picture Association of America. Now Hood has a new bone to pick with the search giant. Yesterday, Hood filed a lawsuit (PDF) against Google in Lowndes County Chancery Court, saying that the company is gathering personal data on students who use Google's G Suite for Education, (previously called Google Apps for Education). In a statement, Hood said that "due to the multitude of unclear statements provided by Google," his investigators don't know exactly what information is being collected. "Through this lawsuit, we want to know the extent of Google's data mining and marketing of student information to third parties," Hood said. "I don't think there could be any motivation other than greed for a company to deliberately keep secret how it collects and uses student information." The complaint claims that through a child's educational account, "Google tracks, records, uses and saves the online activity of Mississippi's children, all for the purpose of processing student data to build a profile, which in turn aids its advertising business." That gives Google an unfair edge over its competitors and violates Mississippi consumer protection law, say state lawyers. More than half of Mississippi schools use Google products, according to Hood's office. A Google spokesperson declined to comment on the matter. Google said that it stopped collecting any student data for advertising purposes in 2014. In an interview with the Jackson Clarion-Ledger, Hood explained one investigative technique his office used.

The investigators logged on to a laptop with a student's educational e-mail address and password and made some queries on YouTube.

Then they logged out, went to a different browser, and logged in again. "It started shooting ads at us dealing with the same query that that child had put in.
So we knew that they were tracking that child," Hood told the Clarion-Ledger. Hood's concerns mirror those from an Electronic Frontier Foundation complaint over Google Apps for Education, filed with the Federal Trade Commission in late 2015.

That complaint pointed out that Google Chrome's "Sync" feature was enabled by default on educational laptops, meaning that Google could track, store, and data-mine student Internet use, although not for advertising purposes. As of last month, the FTC had yet to take action on the EFF complaint.
EnlargeGetty Images/Urich Baumgartgen reader comments 4 Share this story Online messaging services such as WhatsApp, Skype, and Gmail face a crackdown on a "void of protection" that allows them to routinely track the data of EU citizens without regulatory scrutiny—and it could be bad news for ad sales. On Tuesday, officials in Brussels proposed new measures to curb Silicon Valley players who—up until now—have been largely immune from the ePrivacy Directive, which  requires telecoms operators to adhere to the rules on the confidentiality of communications and the protection of personal data. As part of its planned overhaul, the European Commission, the executive wing of the European Union, said that it planned to beef up the measures by switching from a directive to a "directly applicable regulation" to ensure that the bloc's 500 million citizens "enjoy the same level of protection for their electronic communications." It claimed that businesses would also benefit from "one single set of rules." Over-The-Top services such as Facebook's WhatsApp and Google's Gmail can all but ignore the EU's existing rules.

The commission said that this needed to change: Important technological and economic developments took place in the market since the last revision of the ePrivacy Directive in 2009.

Consumers and businesses increasingly rely on new Internet-based services enabling inter-personal communications such as Voice over IP, instant messaging, and Web-based e-mail services, instead of traditional communications services... Accordingly, the Directive has not kept pace with technological developments, resulting in a void of protection of communications conveyed through new services. The EC is also planning to kill the heavily ridiculed cookies consent pop-up system.
It said, in an embarrassing—if long overdue—climbdown that users would be given more control to allow or prevent websites from tracking them depending on "privacy risks." Last summer, a big coalition of tech firms lobbied for the cookie law to be scrapped. Under the new proposal, the commission said: "no consent is needed for non-privacy intrusive cookies improving Internet experience (e.g. to remember shopping cart history).

Cookies set by a visited website counting the number of visitors to that website will no longer require consent." But it could also hit the bottom line of Facebook, Google, and chums because tracking consent may be harder to obtain if lots of users reject third party cookies.

The commission said that, following public consultation on the issue, 81.2 percent of citizens agreed that obligations should be imposed on "manufacturers of terminal equipment to market products with privacy-by-default settings activated." It also warned that "additional costs" could hit some Web browser makers because they would be required to develop software with privacy settings built in. The new proposals also call on consent to process electronic communications metadata, such as device location data to allow for the "purposes of granting and maintaining access and connection to the service," the commission said.
It means that telcos "will have more opportunities to use data and provide additional services." Translation: new ways to make more cash. Companies that flout confidentiality of communications rules face fines of up to four percent of their global annual turnover, under the commission's planned e-privacy measures—the same penalty that will be dished out to firms that violate the EU's General Data Protection Regulation, which comes into action in April 2018. "The European data protection legislation adopted last year sets high standards for the benefit of both EU citizens and companies," said EC justice chief Věra Jourová. "Today we are also setting out our strategy to facilitate international data exchanges in the global digital economy and promote high data protection standards worldwide." But the latest proposals cannot become law until the bloc's 28 member states and the European Parliament agree to wave them through—leaving plenty of wiggle room for industry lobbying. Separately, the commission is seeking views from the public on how to best tackle data mining as part of its Digital Single Market strategy. This post originated on Ars Technica UK
With digital transformation dominating the business agenda, IT pros are under pressure to create a modern-day tech foundation sturdy enough to drive that change as they head into 2017. What milestones are they aiming for in the year ahead? Where should they direct their limited resources? According to Computerworld’s Forecast 2017 survey, IT professionals will prioritize security, analytics, XaaS or “as a service” technology, virtualization and mobile apps in the coming year.
If you’re thinking of adding those technologies to your own 2017 to-do list, read on for findings from our survey, along with real-world advice from other IT leaders. 1.
Security High-profile corporate data breaches, politically charged cyberattacks like those against the Democratic National Committee (DNC), and the October DDoS attack that took down much of the internet have kept security front and center this year, prompting many in IT to ramp up strategies and add layers to their lines of defense. In Computerworld’s Forecast 2017 survey, 47 percent of the 196 IT professionals polled said they plan to increase spending on security technology in 2017, and 14 percent chose security as the most important technology project currently underway at their organizations. Moreover, 15 percent of those surveyed said they expect security to be their top leadership challenge over the next 12 months, and another 15 percent said they’re currently beta-testing enterprise security technologies.

Even those not pursuing specific security initiatives have security on their minds: 19 percent of the respondents said their primary goal for their most important project is “to meet security, privacy or compliance goals.” At Global Strategy Group, a New York-based public affairs and political research firm, security is always a priority, but it’s at the very top of the list this election year. “As a firm that works with the DNC, security has always been a top thought, but when it becomes front page news, everyone is looking at it,” says Andrew Ho, the firm’s vice president of technology.

Global Strategy plans to tighten things up over the coming year with tools like single sign-on and multifactor authentication, but Ho says it’s really less about the technology and more about changing employee behavior. “We’re definitely looking at new technology, but it’s 70 percent about the culture,” he says. “It’s about changing people’s behavior and thinking through the psychology of what really gets hacked.
It’s not necessarily about a stronger firewall, but about people realizing they can’t do things like use the same passwords for everything.” Hiring more people to confront security challenges is another common tactic. Of the Forecast survey respondents expecting to add new employees this year, 30 percent said the reason was to bolster security initiatives, with 26 percent of them saying they anticipate new hires in the areas of security, compliance and governance. They might have trouble finding qualified talent, however: One-quarter of the survey respondents with hiring plans ranked security as the most difficult skill to hire for. 2.

Analytics As companies double down on their efforts to get closer to customers, data has taken on critical importance, with analytics serving as a springboard for success. Organizations are stockpiling data on web traffic, customer preferences, buying behavior, real-world product performance and more, creating a potential gold mine of insights—if they adopt the right strategy and use the right analytics tools to make sense of everything they collect. Some 38 percent of survey respondents said they plan to increase spending on data analytics (a category that includes big data, enterprise analytics, data mining and business intelligence tools) next year, and data analytics was No. 4 on the list of technology projects that respondents cited as their organizations’ most important initiatives. Moreover, 21 percent of respondents said their organizations are engaged in a beta test of a big data project, and nearly 30 percent pegged big data/analytics as the disruptive technology most likely to impact their organization over the next three to five years. In Edmonton, Alberta, the city government has made analytics an organizational focus for 2016 and 2017.

The plan, currently underway, is to build a federated-style data warehouse linked to data marts, leveraging big data tools like Tableau for visualization. “The goal is to break down barriers and let previously siloed information be available and useful to others to enable better decision-making,” says Bruce Rankin, service support lead for the municipal government’s Spatial Centre of Excellence. Global Strategy Group’s Andrew Ho It’s a similar story at Global Strategy Group. Over its 20-year history, the firm has collected tons of data on elections, and it’s using analytics to identify trends, says Ho. The next step is to take open technologies like R, a programming language tuned for statistical computing, and visualization capabilities like Tableau and remake static PowerPoint presentations into something dynamic that can reflect trends in polling. “We want to put an iPad in front of a client, change a variable here or there, and see how the trends change,” he says. “It’s a practice we started this year.” 3. XaaS Another year in and there’s no stopping the cloud computing juggernaut, especially as companies retool IT infrastructure for digital transformation. The “as a service” trend continues to gain traction, with 33 percent of survey respondents reporting that their organizations are planning to increase spending on software-as-a-service (SaaS) offerings next year, putting SaaS at No. 5 on the list of respondents’ most important technology projects. At the same time, 24 percent of those polled said they intend to spend more on platform-as-a-service (PaaS) technologies and 27 percent said they will put more money toward infrastructure as a service (IaaS) in 2017.

And finally, 29 percent of respondents expect cloud or SaaS systems to be the disruptive technology that has the most impact on their business over the next three to five years. As interest in hosted systems gains steam, 13 percent of the respondents said they’re beta-testing SaaS offerings, while 12 percent are taking PaaS for a spin and 8 percent are piloting IaaS. With all of that cloud activity underway, IT leaders are looking to expand their teams’ skill sets: 26 percent of survey respondents who plan to increase head count in the next 12 months said they intend to hire people with cloud and SaaS skills, putting cloud/SaaS expertise at No. 5 on the list of skills they’re seeking. Roche Bros., a 20-store Boston-area grocery chain, is moving as much of its infrastructure and applications as possible to the SaaS model, says John Lauderbach, the company’s vice president of IT. Human resources applications, sign and tag printing systems, backup and recovery tools and even mainstream productivity applications have all been moved to the cloud, helping to reduce costs and provide better reliability and 24/7 availability, according to Lauderbach. “We are four people [in IT], so for us to manage servers, facilitate patches and do backup and recovery work is a lot,” he says. “There are people who do this for a living who can do it better than we can.” 4. Mobile apps As smartphones and tablets become standard fare for consumers and employees alike, IT groups are racing back to the drawing board to retool existing applications to be mobile-friendly while creating new mobile apps to court customers and gain competitive advantage. In our Forecast 2017 survey, 35 percent of those polled said they plan to increase spending on mobile systems next year. Nearly 10 percent said they’re beta-testing mobile apps, while 21 percent of those with hiring plans said they hope to add people with mobile application and device management skills. Mott Community College’s Cheryl Shelton Mott Community College in Flint, Mich., has a mobile app to facilitate student services, but it’s also in the process of revamping its website to make it more mobile-friendly.
Students want access to critical information on their go-to mobile devices, though they aren’t married to the idea of a single-purpose app, says CTO Cheryl Shelton. “We decided to go with an adaptive web design because an app doesn’t fit our culture,” she says. “We decided to build a robust website and make that work for mobile instead of limiting ourselves and wasting a lot of time keeping an app up to date.” 5.
Virtualization The march toward wholly virtualized IT environments rolls on.

Companies are virtualizing more than just desktop systems these days and are beginning to expand their efforts to areas like servers, networks, storage and even mobile infrastructure. Some form of virtualization will be on the docket for 29 percent of survey respondents in 2017, and of those who are planning hiring increases next year, 18 percent said they will be looking for people with expertise in virtualization. Desktop systems are still the most common targets of virtualization initiatives—16 percent of respondents to the Computerworld Forecast 2017 survey said they currently have desktop virtualization beta tests underway.
Storage is the second most common technology to be virtualized, with 11 percent of respondents saying they’re beta-testing such systems, followed by server virtualization (being beta-tested by 10 percent of those polled), mobile virtualization (8 percent) and network virtualization (7 percent). Roche Bros. has already gone all-in with virtualization, using the technology for desktops, servers, networks and storage, says Lauderbach. “Everything we have is already virtual,” he says. Similarly, the Edmonton city government has been pursuing infrastructure virtualization projects for years, says Rankin. “Virtualization keeps going and growing, but it’s not at a point anymore where it’s strategic—it’s just what we do,” he says. The year of the customer IT leaders are adopting and applying these five technologies with specific business outcomes in mind. One of the most important is customer satisfaction. Nearly half (48 percent) of the respondents to the Computerworld Forecast 2017 survey said improving customer satisfaction or the customer experience was the most important business priority for IT in the coming 12 months. That makes sense to Global Strategy Group’s Ho, whose goals for 2017 include improving the customer experience. “IT is here to serve the business; our business isn’t IT,” he says. “Customers and users are the ones that need to be happy and able to do their work well.

The way that happens is that we make their experience as great as possible.” This story, "5 key technologies to double down on now" was originally published by Computerworld.
Watson is done with school -- for now -- and is ready to try out what it has learned in the real world. IBM has launched the Watson for Cyber Security beta program to encourage companies to include Watson in their current security environments.
Modernised system enables public safety agencies to share and analyse critical evidence while integrating social media and allowing citizens to provide information and track requestsLondon, October 18, 2016 – Unisys Corporation (NYSE: UIS) today announced the availability of the Unisys Digital Investigator™, a new total information management system to allow law enforcement agencies to seamlessly share critical investigative intelligence across applications and agency boundaries. The system, an upgraded version of a solution previously known as the Unisys Law Enforcement Application Framework, includes a public portal which enables citizens to provide information to law enforcement and public safety agencies and track requests from smartphones and other devices. It also integrates with social media analytics to assist in compiling digital evidence. The Unisys “Safe Cities” initiative provides tools and capabilities that can help prevent and combat crime. As a key tool in the Safe Cities portfolio, Digital Investigator goes beyond the capabilities of traditional records management systems to allow organisations to take advantage of information across technological and geographic boundaries. It provides the tools necessary to capture and analyse the increasingly varied and complex data law enforcement professionals need to catch criminals and stop criminal gangs and terrorists.“The Unisys Safe Cities initiative was created to give justice and public safety organisations the solutions they need to better use data from sources like mobile devices and the Internet of Things to create a safe environment for the public,” said Mark Forman, global head of public sector at Unisys. “Digital Investigator represents a critical component of this initiative. It gives police departments a tool set that improves productivity through the ability to handle numerous forms of data across multiple lines of inquiry while linking different pieces of related information.” When it comes to criminal investigations and intelligence handling, the use of data mining to identify suspects early has emerged as a critical challenge. With so much information available from different sources, law enforcement agencies are grappling with how best to collect, collate and analyse all pieces of evidence, including their movements through the justice system as well as their storage and destruction. Digital Investigator addresses this challenge by providing the capabilities needed to efficiently and successfully manage all police incident and investigative information, including: A device-independent public portal for information capture and delivery, enabling citizens to connect to law enforcement agencies to report missing persons, request appointments and receive updates to their requests; Social media analytics integration to stay on top of internet-derived intelligence from a vast array of sources; A single, browser-based application for total information management, tasking, collaboration and analytics; An easy-to-use and cost-effective tool for creating reports and other documents; Advanced search engine capabilities for rapid and relevant results; Certified ability to securely handle critical national security data; and On-premise and cloud-based deployment options. Digital Investigator is part of Unisys’ portfolio of Safe Cities solutions that also include LineSight™ for advanced targeting analytics related to border security; Unisys Stealth® for advanced security through micro-segmentation as well as identity management; the Unisys Law Enforcement Message Switch for enabling users to easily access disparate information from various federal and state criminal justice information systems; and the Secure Image Management Solution for capturing and aggregating stills imagery, video, audio, multimedia and disk images. About UnisysUnisys is a global information technology company that works with many of the world's largest companies and government organisations to solve their most pressing IT and business challenges. Unisys specialises in providing integrated, leading-edge solutions to clients in government, financial services and commercial markets. With more than 20,000 employees serving clients around the world, Unisys offerings include cloud and infrastructure services, application services, security solutions, and high-end server technology. For more information, visit www.unisys.com. Follow Unisys on Twitter and LinkedIn. Contacts:Brad Bass, Unisys, 703-439-5887brad.bass@unisys.com UK: Jay Jay Merrall-Wyre, Unisys, +44 (0) 20 3837 3729unisys@weareoctopusgroup.net ### Unisys and other Unisys products and services mentioned herein, as well as their respective logos, are trademarks or registered trademarks of Unisys Corporation. Any other brand or product referenced herein is acknowledged to be a trademark or registered trademark of its respective holder.
Step two: Keep doing that While venture capitalists have been tightening their belts over the past year, there’s still a lot of love and funding for security startups – especially if you’re working in the right areas. During a panel discussion at the Structure Security conference in San Francisco today, a trio of top VCs identified three key areas where security startups would have no problems getting initial funding: cloud security, containerized protection, and machine learning – although that last area comes with caveats. Asheem Chandna, a partner at venerable VC firm Greylock Partners, and Theresia Gouw, managing partner at Aspect Ventures, both said they were investing heavily in cloud security outfits, since they matched the tech industry's ongoing movement to the cloud and the need for increased security budgets. “In the last two decades that I’ve been doing this there never been a better time to be in this [security] business,” Chandna said. “Look at how important it is to buyers and how the checkbooks are open. Most people expect security budgets to double in the next few years – it’s a good time to be an entrepreneur in the cloud space, but for customers it’s confusing.” The second area of interest among funders is containerization, and how it can be secured and used to protect data and applications.

Alex Doll, founder of Ten Eleven Venture, said it was an area his firm was spending a lot of time and money on. “When we look at containers it's as big a trend as Linux was a decade ago, or virtualisation was a few years ago,” he said. “We think the containers trend is a series-A level for investment funders.” Machine learning for security was also a hot area, he said, but warned it required care to train up AI models with high quality information – to sort the wheat from the chaff, in other words.

True machine learning systems in security are rare, and too many startups claim they are applying AI techniques when, in fact, they are simply running human-overseen data mining, and often coming to the wrong conclusions. Chandna said that VC funding for the broader IT area is shrinking slowly, as VCs winnow out the “me-too companies” and those without a realistic growth plan.

For security, there is still a lot of startup moolah out there for budding entrepreneurs to tap. But before you tell your boss he or she's a pillock and rush out to set up your own firm, be warned. While getting initial series-A funding is relatively easy, getting more funds out of VCs is getting a lot harder. “A-level funding is still easiest to raise,” Chandna said. “Series B is easier but valuations have come down.

But when it comes to Series C or D then firms need to show real progress and customer wins.

The cost of capital has gone up.” ®