Cloud Innovators Create a Body and Soul for Big Data

brain cap“It’s alive! It’s alive!” Thus spoke Dr. Henry Frankenstein of his creation in the iconic horror film classic from 1931, as does the main character in Mel Brooks’ 1974 comedy Young Frankenstein. It’s Alive is also the title of a little known 1974 B horror movie about a murderous infant. Can the same be said of computers or the data and information that inhabits them?

In June 2014, a computer program, through text-based conversations, is purported to have become the first to pass a Turing Test by fooling 33 percent of humans into thinking it was a 13-year-old Ukrainian boy. The test was devised in 1950 by computer-science pioneer and World War II code breaker Alan Turing, who said that if a machine was indistinguishable from a human, then it was “thinking.” Granted, some doubt the efficacy of these test results.

Of course, the idea that humankind, by dint of desire, intellect and ingenuity, can create life from inanimate objects is not a new concept. Almost 200 years ago, Mary Shelley, a talented young English lass barely out of her teens, bestowed life on the original Frankenstein monster and its overzealous creator, spawning a series of spinoffs and imitations as a result.

An illuminating quote from Shelley’s book reads, “So much has been done, exclaimed the soul of Frankenstein – more, far more, will I achieve; treading in the steps already marked, I will pioneer a new way, explore unknown powers, and unfold to the world the deepest mysteries of creation.”

In his Pulitzer Prize–winning book, The Soul of a New Machine,Tracy Kidder refers admiringly to one of his subjects, Carl Alsing: “That was what made it fun; he could actually touch the machine and make it obey him.” Said Alsing, “I’d run a little program and when it worked, I’d get a little high, and then I’d do another. It was neat. I loved writing programs. I could control the machine. I could make it express my own thoughts. It was an expansion of the mind to have a computer.”

One of Think and Grow Rich author Napoleon Hill’s most enduring quotes is: “Whatever the mind can conceive and believe, it can achieve.” Couple that thought with man’s tendency to anthropomorphize any other living or lifeless entity, and it is not much of a stretch to imagine computers, software or data imbued with a “living” body – and a soul.

Anatomy of Big Data: Body and Soul, Quantitative and Qualitative

Perhaps it is man’s destiny, or greatest conceit, to remake virtually everything in his/her own image and, along the way, define the essence or animating principal of life. Aristotle in Book II of De Anima (On the Soul) describes soul as “the essential whatness of a body.” He adds, “The soul is inseparable from its [whole, living] body”; and “What has soul in it differs from what has not in that the former displays life.” He defines “life” as including thinking, perception, movement, self-nutrition, decay, growth and the power of sensation.

Most all of the world’s religions recognize soul or anima – Latin for the animating principle, or psyche in Greek – regardless of whether they view only humans as possessing immortal souls or they believe, as do pantheists, that even inanimate objects such as rivers and mountains have souls and are “living.”

Tu Weiming is Director of the Institute for Advanced Humanistic Studies at Peking University and Research Professor and Senior Fellow of Asia Center at Harvard University. On the subject of “human rootedness” of Confucian thought, he identifies three characteristics, including Cheng (juhng), the state of absolute quiet and inactivity; Shen (shen), which concerns the “heavenly aspect of the soul” and its development; and Chi (jee), an “originating power, an inward spring of activity … a critical point at which one’s direction toward good or evil is set” can be identified and used to further “flourish the soul.”

Qualtism or Kwalitisme is defined as “the search for the soul in everything that surrounds us in this over and over quantified society.” The ever-expanding universe of Big Data is akin to a Mulligan stew of ingredients culled from a vast variety of sources – most are accessible through the web – from which we seek to derive structure, meaning and insight, limited only by our lack of imagination and our unwillingness to change.

Descriptive terms such as agile, aware, elastic, innovative, intelligent, intuitive, sensitive are attributed to both hardware and software solutions, further blurring the lines between animate and inanimate objects. In the recently released film by Spike Jonze, Her, Joaquin Phoenix falls in love with an intelligent OS (operating system) that simulates a woman’s voice – no doubt, Apple’s Siri was an inspiration.

Biomedical devices replace lost limbs and organs or, in the case of cardiac pacemakers, provide electrical impulses to keep the heart pumping while simultaneously collecting and sending data to care providers. The MoMeTM System from Infobionic is “the first Cloud-based, universal patient-monitoring solution with unprecedented analytics that allows physicians to quickly and accurately diagnose and treat patients.”

Wearable “intelligent” devices are flooding the market, including smart watches that track fitness; medical devices that keep track of glucose levels and blood pressure; smart clothing and eyewear that gather and aggregate data; wearables for babies, kids and pets, all available online through Amazon and other sources.

Meanwhile, an article, Soul Searching or Data Mining; Distinctive Pathways to Health, written for a Christian Science publication questions the overuse of technology and its implications for spirituality in healthcare. “As healthcare continues to be increasingly linked with technology, data, and endless quantification, staying connected to our soulful self and reflecting upon our inner identity can bring healthy benefits. After all, all the numbers in the world don’t begin to tell the whole story of you.”

Just as digital networks and power grids feed trillions of data points to centralized computers, the body feeds data to the brain – and now increasingly to external computing devices via the Internet. Humans are truly becoming part of the Internet of Things as our bodies are in the process of becoming primary contributors to the Big Data pool. Progressively, we are being connected to and relying on intelligent devices for a variety of purposes – despite concerns about the loss of our humanity. But isn’t it human to want to create humanlike things?

Big Data and the Emergence of Cognitive Computing

Early on, computers mastered deterministic or predictive computing, commonly defined as the ability to objectively predict an outcome or result of a process due to knowledge of a cause-and-effect relationship – such as adding or subtracting numbers. In the 1970s, relational databases emerged as a tool to manage text in a much more predictable fashion, using rows and columns containing, for instance, customer names, addresses, age, sex and other structured demographic information.

A friend and former colleague of mine, Dr. Adrian Bowles, an expert on artificial intelligence and founder of STORM Insights, has posted a six-minute long You Tube video describing neuromorphic architectures and defining cognitive computing, which is how computers “learn” to think like humans.

Below is one of Bowles’ presentation slides.

Adrian Bowles Storm Insight Cognitive Computing Framework

According to Bowles as well as an article published earlier this year by MIT Technology Review, chipmaker Qualcomm is planning to deliver a neuromorphic chip next year that will enable even smarter smart phones with sensory inputs to see, feel, read and predict outcomes.

The MIT article claims, “These ‘neuromorphic’ chips – so named because they are modeled on biological brains – will be designed to process sensory data such as images and sound and to respond to changes in that data in ways not specifically programmed. They promise to accelerate decades of fitful progress in artificial intelligence and lead to machines that are able to understand and interact with the world in humanlike ways.”

In a 2011 presentation given by two undergraduate students, James Kempsell and Chris Radnovich from Rochester Institute of Technology (RIT), on Neuromorphic Architectures, the pair concludes: “Neuromorphic Architectures will be the next major step after Von Neumann. These architectures will help realize how to create parallel locality-driven architectures – used for what the brain is good at: compressing data into information.”

Here is one of the more interesting slides from the RIT Students’ presentation.

Neuromorphic Architectures explained slide

The students also quote Dr. Leon Chua of UC Berkeley, who more than 40 years ago first envisioned the existence of the memristor, or “memory resistor,” which digitally and mechanically mimics the biology of the human brain. “Since our brains are made of memristors, the flood gate is now open for commercialization of computers that would compute like human brains, which is totally different from the Von Neumann architecture underpinning all digital computers.”

As stated on the IBM Research website, in August 2011, as part of IBM’s SyNAPSE Project (Systems of Neuromorphic Adaptive Plastic Scalable Electronics) project, “IBM researchers led by Dharmendra S. Modha successfully demonstrated a building block of a novel brain-inspired chip architecture based on a scalable, interconnected, configurable network of ‘neurosynaptic cores’ that brought memory, processors and communication into close proximity. These new silicon, neurosynaptic chips allow for computing systems that emulate the brain’s computing efficiency, size and power usage.”

Not to be outdone, HP unveiled the Machine at its Discover 2014 Summit in June. The genesis of the Machine is a project led by Dr. Stan Williams, an HP Senior Fellow and director of the Memristor Research group at HP Labs, which began in 2008. As reported by HP, “Williams is currently focused on developing technology that supports the concept of CeNSE: The Central Nervous System for the Earth. The idea is that nanotechnology has the potential to revolutionize human interaction with the earth as profoundly as the Internet has revolutionized personal and business interaction.”

While Quantum expects its neuromorphic chips to be generally available sometime in 2015, IBM’s chip and associated “new programming language to enable the development of new sensory-based cognitive computing applications” is still a few years away from mainstream use, as is HP’s Machine. In the interim, the next 5 years or so should bring some potentially transformational changes in the standard computing architecture model, aka Von Neumann’s approach and chip fabrication.

Consider that neuroscientist Christof Koch lists the total synapses in the cerebral cortex at 240 trillion (Biophysics of Computation. Information Processing in Single Neurons, New York: Oxford Univ. Press, 1999, page 87), while IBM’s SyNAPSE project hopes to reach 100 trillion – at some point in the near future with its “human-scale” simulation.

“From a pure energy perspective, the brain is hard to match,” says Stanford University bioengineering professor Kwabena Boahen who has developed Neurogrid chips to simulate brain function. “The human brain, with 80,000 times more neurons than Neurogrid, consumes only three times as much power. Achieving this level of energy efficiency while offering greater configurability and scale is the ultimate challenge neuromorphic engineers face.”

Big Data Leaving the Predictive Past Behind

Accompanied by the advent of Big Data, much of it in less structured formats such as text in documents, emails and webpages or data derived from images, video, voice or machine data, is a strong desire to plumb the depths of that data for new insights. Hence, computer scientists and engineers are redoubling their efforts to reformulate the predominant computer model to replicate the way people think or, at the very least, predict how people will behave.

Most predictive models rely on past behavior or a retrospective approach. It’s ingrained in us that “People don’t change” or that “Change is hard.” And so it is with predictive models. However, a predictive model that relies on the fact that I have made 30 roundtrips on the same airline to Chicago in 18 months but for the last 6 months, I have not flown on that airline once is cause for predictive modeling confusion. Was I unhappy so I changed my carrier? Was I visiting a friend or relative who has since moved? Did I switch jobs and now have a new territory? Have I retired or expired?

Companies such as Acxiom sell data about millions of people to thousands of companies looking to cross-reference data points and answer questions like those raised above. Healthcare Insurance and Financial Services companies buy data in order to fill out customer profiles – what they refer to as a “Customer 360” view. However, as many data wonks have come to realize, correlation does not imply causation.

Meanwhile, there are scores of new solutions that support cognitive and probabilistic computing models. Like humans, cognitive computing applications, also referred to as artificial intelligence, learn by experience and instruction, then they adapt. For instance, take a solution such as IBM’s Watson, which functions “by understanding natural language, generating hypotheses based on evidence and learning as it goes.”

Like a human, Watson and other cognitive solutions have intent, memory and foreknowledge, while they mimic human reasoning within specific specialized domains with variable outcomes – if trained by experts in the field. Healthcare providers use Watson to support physicians to arrive at difficult diagnoses. Unlike humans, Watson can quickly search through millions of records to find patient records that refer to common symptoms – called similarity analytics – and search through medical ontologies and thousands of research papers to provide physicians with weighted recommendations and several probable diagnoses.

Wall Street uses Monte Carlo methods to value and analyze complex investments by simulating thousands of random or “probable” sources of uncertainty and then determining the average value based on the range of results. Monte Carlo simulations are often used for a variety of other types of institutional and personal investments, such as modeling the future performance of a 401k investment or fixed-income instruments. Yet, Monte Carlo simulations are limited for predicting human behavior – unless you count automobile traffic-safety assessments and perhaps some other niche applications.

Big Data Needs a Soul

“Data without a soul is meaningless,” writes Om Malik of GigaOm in a post from March 2013. “Empathy, emotion and storytelling – these are as much a part of the business as they are of life. The problem with data is that the way it is used today, it lacks empathy and emotion. Data is used like a blunt instrument, a scythe trying to cut and tailor a cashmere sweater.

Malik adds, “The idea of combining data, emotion and empathy as part of a narrative is something every company – old, new, young and mature – has to internalize. If they don’t, they will find themselves on the wrong side of history. What will it take to build emotive-and-empathic data experiences?”

Malik as well as others suggest the ability to ask human questions, to be data-aware and data intelligent and to have the facility to make correlations between data, in context, is where the future of computing ultimately needs to head.

In his recent post, The Soul-Crushing Problem with Data, Andrew Eklund makes the case that “the more and more we rely upon data to inform decisions, the less we often trust our most basic of human instincts – our gut.”

Eklund adds, “We’ve become so heavily dependent upon data that the creative process becomes hijacked from the outset. For example, there’s a big difference between ‘What does the data say about a consumer?’ and ‘What do we think the consumer needs or wants?’ The problem is that the data will rarely answer the second question, even though that is our greatest challenge.”

Despite his concerns, Eklund is a strong advocate for data in general: “Don’t get me wrong. I love data. Data is the gift that keeps on giving with regards to ongoing insights, measurement of performance, targeting and re-targeting, campaign optimization, and a thousand other amazing things wrapped up in jargon we love and overuse. But data isn’t an idea. An idea comes from within your soul.

Matt Ballantine’s equally emotionally charged blog, Big Data: The Tyranny of the Past, suggests that “Big Data might be quite good at predicting the future (most of the time) in comparison to humans because Big Data will predict based on trends. If we could put the issue of selection bias aside, that then might make for better decision making. Except when the unexpected occurs, at which point Big Data extrapolating trends will fall on its backside. And here’s the other big challenge for Big Data – that it’s great at spotting correlations but does nothing to understand causality.

Re-imagining Big Data

Sean Gourley, Co-Founder and CTO of Quid, an intelligence amplification platform, states: “Data science, I believe, we need to re-imagine it because data is incredibly powerful. We need to step back from the scientific notations and start thinking of it as data intelligence. Data intelligence has a slightly different philosophy that embraces some of the messy and unstructured nature of the world that we do live in.”

As Malik states in his aforementioned post, “Data needs stories. The symbiotic relationship between data and storytelling is going to be one of the more prevalent themes for the next few years, starting perhaps inside some apps and in the news media. In a future where we have tablets and phones packed with sensors, the data-driven narratives could take on an entirely different and emotional hue.”

Data in and of itself has no meaning or purpose. Data needs a human, qualitative, soulful element to enrich and enhance the quantitative component that is inherent in all static, lifeless forms of data.

Cloud: The Fundamental Building Block for Humanizing Big Data

Big Data also needs an ample, expandable body or elastic infrastructure to contain zettabytes (for now) of data. The closest thing we have today is the Cloud, aka the Internet that, in theory, encompasses all publicly available electronic data as well as data behind firewalls and contained in private networks, or private Clouds, often available to users via the Cloud.

NIST defines Cloud computing as “a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.”

Research firm IHS predicts worldwide spending on Cloud computing will reach nearly $250 billion by 2017, while in 2014, global business spending for infrastructure and services related to the Cloud will reach an estimated $174.2 billion. IDC, in its Cloud Forecast for 2014, predicts, “the Cloud software market will surpass $75B by 2017, attaining a five-year compound annual growth rate of 22% in the forecast period.”

cloud-spending-soars

The Cloud phenomenon touches every business, regardless of size, and virtually every smart phone, personal tablet and device. Until such time that personal devices can store trillions of data points, the Cloud is our best route to humanizing big data. As such, the Cloud, like man, is evolving, requires care and tweaking, and needs to learn, grow and be protected from those that would do it harm.

Bottom Line

The primary Cloud infrastructure components (the Body) are Compute, Storage and Networks – and, one could argue, Databases, various forms of Security and the ability to self-replicate. Extending the metaphor, software applications orchestrate the movement of data and, more and more, re-purpose data to learn, think, reason, perceive and motivate – which some might define as the Soul, albeit one with mostly digital roots.

Regardless of one’s beliefs, the fact is that machines are getting smarter in ways that people have traditionally measured human intelligence – from a quantitative perspective. Adding qualitative, cognitive computing capabilities changes the entire thinking around the usefulness of computers – for better or worse.

Big Data, loosely defined as the sum of all electronic data and information, accompanied by advances in Cloud computing, chip fabrication, natural language processing and machine learning are all in the process of transforming the utility of computing and challenging man’s traditional view of what constitutes “life” as we know it.

 

Appendix: 38 Big Data and Cloud Innovators to Look for in 2015

To be sure, there are literally hundreds of companies and thousands of individuals contributing to the evolution of the Cloud in some way, shape or form, accompanied by those delivering innovative solutions for the management of Big Data.

What follows are several examples of Big Data and Cloud innovators as well as a few taxonomy slides to help define different categories of solution providers. Representative solution and service providers include large multinationals, established midsize vendors and veritable start-ups, with a primary focus on Big Data and Cloud Computing.

* For readers interested in reviewing Cloud implementation approaches being tested and implemented by large global banks and financial services firms that will hopefully benefit the entire FS industry, and perhaps other industries as well, click on the following link. (FYI, Parity Research is a co-founder of the Cloud Open API Forum for Financial Services.) IaaS and PaaS: Mature Enough for Financial Services Firms? Vendor sponsors for the FS Cloud Forum include ActiveState, Citrix, Cloudscaling, Canonical, GigaSpaces, IBM/Softlayer, Mirantis, MuleSoft, SolidFire, SunGard Availability Services and SwiftStack.

A high-level taxonomy for Cloud Infrastructure as envisioned by IBM follows:

cloud taxonomy from IBM

NIST’s visual model or taxonomy for Cloud infrastructure follows:

cloud taxonomy from NIST

* For additional Cloud taxonomies, readers can review the Cloud Blueprint Blog authored by Ravi Kalakota.

IaaS – Infrastructure as a Service Providers

The IaaS category is packed with solution providers, from regional players that focus on specific industries such as midsize banks to large multinational providers with services that literally span the globe. Other providers specialize in supporting specific application types such as Microsoft applications and databases. Below are several representative examples.

IaaS appeals to companies of all sizes who are looking to lower the cost of their IT infrastructure. However, most large companies are reluctant to wholly embrace IaaS due to concerns that include security, availability, compliance and vendor lock-in. Vendors such as Amazon and IBM contend their IaaS offerings are more secure than their customers’ data centers. From an infrastructure standpoint, this very well may be true.

Meanwhile, most large organizations are experimenting with several IaaS providers at once, if even on a limited basis, because the potential cost, backup and disaster recovery, elasticity and, perhaps at some point, the performance and agility aspects of Cloud computing are just too compelling to ignore. Competition also induces providers to lower prices and offer incentives.

Note: Many larger IaaS solution providers offer services and capabilities across the spectrum of Cloud products, while other IaaS providers (e.g. Canonical and Cloudscaling) do not maintain their own data centers but leverage the expertise and assets of other providers.

AWS is Amazon’s industry-leading web services solution set which provides a collection of computing services that together make up their Cloud computing platform, including Amazon EC2 and S3 services. AWS has a head start on every other provider in the industry, having had the vision to begin development earlier than most of the competition. Beginning with small companies or development groups within large corporations, AWS now is focused on embracing the needs of large organizations within specific industries such as banking and financial services. While virtually every financial services company has, at minimum, a limited relationship with AWS, Amazon will need to convince the industry that it can meet their needs without sacrificing security, performance and platform agnosticism. At this point, AWS also has the largest partner ecosystem.

AWS Global Infrastructure

 

Canonical believes in the power of open source to change the world. “Canonical was created alongside Ubuntu to help it reach a wider market. Our services help governments and businesses the world over with migrations, management and support for their Ubuntu deployments. Together with our partners, we ensure that Ubuntu runs reliably on every platform, from the PC and the smartphone to the server and, crucially, the Cloud.” Ubuntu could not exist without its worldwide community of voluntary developers who also believe open source is critical to the Cloud development. “We are committed to creating it, refining it, certifying it for reliability and promoting its use. Ubuntu Server 14.04is the ultimate enterprise Cloud platform, both for building OpenStack Clouds and for running on public Clouds.”

canonical_ubuntu_release_roadmap

 

Cloudscaling believes “companies across industries will require an open source IaaS solution to cost-effectively support a new generation of Cloud-native workloads and enable the DevOps model – in a multi-Cloud world. That means forward-looking organizations who are building out their Cloud computing footprint require an Infrastructure-as-a-Service solution (IaaS) that is fully interoperable – not just compatible – with the leading public Cloud services.” Its Elastic Cloud Infrastructure built on OpenStack, “enables any IT group to deploy Cloud services comparable to the capabilities of the world’s largest and most successful public Clouds.” Cloudscaling offers its clients increased agility, less complexity and improved time to market, while promoting business and IT alignment.”

Cloudscaling ocs-overview

 

HP, through its Helion portfolio of Cloud products and services, provides an open ecosystem with a common management structure and “integrates easily into your business through a wide range of delivery models offering an open, secure, scalable and agile Cloud environment.” Based on OpenStack, Helion’s portfolio covers the entire spectrum of hardware, software, and professional services. “Architected to work together, the stack is comprehensive no matter how you choose to build or consume your Cloud.” HP has the advantage of a large installed base of solutions that should translate well to the Cloud environment, including its Vertica analytics database, which is “fully provisioned on the Amazon EC2 or any VMware-supported platform and ready for loading within minutes.”

HP Helion Stack

 

IBM, since 2007 when it began working with clients on Cloud computing, has been focused squarely on making the model viable for enterprise and government clients that cannot compromise on security, compliance and availability. “IBM’s strategy for Cloud is clear: We will build Clouds for enterprise clients, and we will provide Cloud services where there are gaps we can fill.” IBM is rounding out its Cloud products and services portfolio through internal R&D and key acquisitions such as IaaS provider Softlayer, their PaaS BlueMix and DBaaS Cloudant committing over $2 billion in the process. IBM’s recently announced Watson Developer Cloud will offer “the technology, tools, and APIs that companies need to develop and test their own cognitive applications, powered by IBM Watson’s cognitive computing capabilities.”

IBM smartcloud

 

Microsoft Azure is winning Cloud-based business, especially with its install base, due to effective marketing and “extremely generous” discounts and incentives. According to a recent Gartner post, “Microsoft’s comprehensive hybrid story, which spans applications and platforms as well as infrastructure, is highly attractive to many companies, drawing them toward the Cloud in general.” Meanwhile, Microsoft runs many of its own popular applications on Azure, including Skype, Office 365, Bing and Xbox. “Azure enables you to build and deploy a wide variety of applications – including web, mobile, media and line-of-business solutions. Built-in Auto Scale features enable you to dynamically scale up and down to meet any needs.” Azure also provides managed SQL and NoSQL data services and support for analytics.

Microsoft Cloud Azure

 

SunGard Availability Services is trusted by companies around the world to keep their IT environments continuously available – including 50% of the Fortune 500, 70% of the Fortune 50 and many others in all industries and sizes. “Though many know SunGard Availability Services as the pioneer of disaster recovery services, we actually provide much more than that today, from highly resilient managed Cloud and hosting for production applications and infrastructure to managed backup, recovery and business continuity services.” Split off from parent SunGard in March 2014, SunGard Availability Services has revenues of $1.5 Billion and 3,000 employees worldwide, while introducing data management and Cloud-recovery services that “significantly reduce recovery times and costs for organizations managing complex, disparate, hybrid environments.”

SunGard AS Enterprise-Cloud

 

Xand provides Hybrid Cloud and co-location services to mostly midsize companies in the Northeastern U.S. “With Xand’s Hybrid Cloud, you can provision Cloud (virtual) and Dedicated (physical) servers on the same network. Our scalable and flexible solution gives you a pay-per-use service that can be quickly added to your environment, giving you scalable infrastructure (servers, storage, security, load balancing and network).” Xand also offers to customers in various industries, including finance and healthcare, Private xCloud with “highly-available facilities, strong levels of compliance, vast engineering resources and 24×7 monitoring. Private xCloud will take your Cloud to new levels of security and stability.” Technology partners include Cisco, VMware, Dell and Red Hat.

Xand Hybrid Cloud

 

Advanced Storage Solutions

This category includes dozens of suppliers of Hybrid Storage as well as solutions that leverage the speed and productivity gains inherent in all-Flash Arrays or Solid State Drives delivered by tech stalwarts Cisco, EMC, IBM and NetApp as well as relative newcomers such as PureStorage, Tegile and Violin. The two examples below are newcomers that have made a commitment to open source Cloud solutions, such as OpenStack and Apache CloudStack, and are focused on the Cloud Service Provider (CSP) market and the enterprise. (For additional information on advanced storage solutions, review Parity’s Flash Memory Summit Blog.)

SolidFire offers the “deepest Cloud integration of any storage vendor. When building large-scale public or private Cloud infrastructures, there is only one choice for block storage. SolidFire delivers the most comprehensive block storage integration with all of the industry leading orchestration software. Each integration surfaces SolidFire’s patent-pending Quality of Service (QoS) controls, allowing for complete performance automation and the development of end-user self-service tools.” SolidFire all-Flash block-based storage arrays are optimized for many open source Cloud solutions, including Citrix, CloudStack and OpenStack. It supplies IaaS providers, including SunGard AS, eBay, Endicia and CenturyLink who are no doubt fans of SolidFire’s QoS, scale-out architecture and low total cost of ownership (TCO) compared with other all-flash solutions.

SolidFire CloudStack

 

SwiftStack focuses on object-based storage applications and is “built on the world’s most popular object store, OpenStack Swift, which powers the largest storage Clouds in the world. SwiftStack already powers the web’s most popular applications that you use every day – and can supercharge your enterprise private Cloud, content storage/distribution, and active archiving applications. SwiftStack places responsibility for the storage system in the software, not in specific hardware components. The SwiftStack Controller manages multiple object storage clusters and removes the heavy lifting from configuration, authentication, cluster management and capacity management. Regular alerts, reports and system stats keep you constantly updated on your storage needs.” SwiftStack collaborates with SolidFire, MongoDB and other open source-centric solution providers on enterprise projects.

swiftstack-architecture

 

PaaS – Platform as a Service Providers

PaaS is a very muddled category as it includes every Cloud middleware, orchestration, automation or enablement solution that cannot logically or easily be called a SaaS (Software as a Service) solution, such as Salesforce.com, or an IaaS provider. Indeed, virtually every IaaS provider has developed its own PaaS layer or is partnering, reselling or is an OEM for an existing PaaS solution. AWS offers its Elastic Beanstalk, and IBM has invested heavily in BlueMix, while HP partners with a variety of suppliers, including open source–based Cloudify and ActiveState, and Cisco with SunGard AS.

A January 2014 Gartner iPaaS MQ specifically focused on enterprise integration PaaS providers includes 17 solution providers with Dell, IBM, Informatica and MuleSoft among the top rated. In a previous research note entitled, Platform as a Service: Definition, Taxonomy and Vendor Landscape, 2013, Gartner identified no less than 15 classes of PaaS, “each roughly mirroring a corresponding class of on-premises middleware products.” By definition, this list would not include PaaS developed exclusively for use by public CSPs such as AWS, Microsoft and others.

Gartner predicts, “Dramatic growth in the iPaaS market over the next five years due to several factors, including:

  • The explosion of CSI [Cloud services integration], MAI [mobile app integration], API and Internet of Things requirements;
  • The emergence of the agile integration approach and citizen integrators, for which traditional integration platforms are unsuitable and;
  • Adoption by SMBs so far often unwilling to embrace integration middleware because of its high cost and complexity, but now interested in iPaaS offerings due to their low entry cost and ease of use.”

The breadth of PaaS offerings is staggering, creating more confusion and uncertainty for enterprises trying to develop their own private Cloud solutions or those enterprises evaluating CSPs to align with for bursting, backup, disaster recovery or Tier 1 application hosting services – all in the name of business agility, application enablement and lower cost.

As with the other categories, the following vendors are a representative example of PaaS solution providers, not an exhaustive list.

ActiveState is the parent company for Stackato, based on Cloud Foundry, which “makes it easy to develop, deploy, migrate, scale, manage, and monitor applications on any Cloud,” available in Enterprise, Micro Cloud, and Sandbox editions. HP Helion began an OEM relationship with ActiveState in late 2012 including Stackato as part of its PaaS portfolio for “building applications for creating private PaaS using any language on any stack on any Cloud. Additionally, enterprise IT can achieve new levels of data security, reduce time to market, save money, ensure compliance, and gain greater control over the Cloud.” Ease of use and agility is achieved through narrowing the gap between development, test and production.

stackato-28-728

 

Citrix CloudPlatform offers “simple, turn-key Cloud orchestration. CloudPlatform is the only Cloud orchestration platform that enables you to quickly and efficiently build a future-proofed Cloud. It is a turn-key solution based on an open and flexible architecture that is designed to run every application workload at scale and with simplicity.” In July 2011, Citrix acquired CloudStack developer Cloud.com, now available through an Apache Software open source license. Considered one of the early Cloud orchestration innovators, Citrix has morphed Cloud.com into CloudPlatform and claims more than 2,500 Cloud providers as users, including AWS. CloudPlatform also leverages standard AWS APIs and a rich partner ecosystem as well as offering its own Turnkey IaaS.

Citrix Cloud Architecture

 

Docker is an “open platform for developers and sysadmins to build, ship and run distributed applications. Consisting of Docker Engine, a portable, lightweight run-time and packaging tool, and Docker Hub, a Cloud service for sharing applications and automating workflows, Docker enables apps to be quickly assembled from components and eliminates the friction between development, QA, and production environments. As a result, IT can ship faster and run the same app, unchanged, on laptops, data center VMs, and any Cloud.” Whether on-premise bare metal or data center VMs or public Clouds, workload deployment is less constrained by infrastructure technology and is instead driven by business priorities and policies. The Docker Engine container comprises just the application and its dependencies.

Docker VM vs Containers

 

GigaSpaces Technologies provides software middleware for deployment, management and scaling of mission-critical applications on Cloud environments through two main product lines, XAP In-Memory Computing and Cloudify. “Hundreds of Tier-1 organizations worldwide are leveraging GigaSpaces’ technology to enhance IT efficiency and performance, from top financial firms, e-commerce companies, online gaming providers, healthcare organizations and telecom carriers.” GigaSpaces recently released its 3.0 version of Cloudify with an open source community version along with a much-improved premium edition that includes an enhanced UI, plug-ins for standard tools, advanced blueprints, elastic caching and Replication as a Service, making it “even easier for enterprises to automate and manage apps and take Cloud orchestration to the next level.”

GigaSpaces Product Portfolio slide

Informatica Cloud’s integration platform-as-a-service (iPaaS) allows enterprises to “extend the capabilities of Informatica Cloud through a variety of development tools, including a REST API and a Java-powered Cloud Connector SDK. Its multi-tenant iPaaS offers benefits for enterprises, including breaking down the walls of data, process and service integration by having a common set of artifacts; increasing collaboration between integration and application developers, as well as ETL and EAI architects; and rapidly connecting to homegrown, proprietary, or legacy systems through REST or SOAP web services, as well as emerging data standards such as JSON or OData.” Informatica iPaaS also offers independent software vendors and system integrators to quickly onboard customer apps and cut the time and cost of integration projects.

Informatica iPaaS

 

Jelastic offers “Platform-as-Infrastructure, the integration of Infrastructure-as-a-Service and Platform-as-a-Service, delivering a scalable, manageable and highly available Cloud.” DevOps can deliver a “private or hybrid enterprise Cloud that drives down costs and increases agility. Provide your developers with an enterprise-class private PaaS that enables rapid application development and deployment without coding to proprietary APIs.” Jelastic automates horizontal and vertical scaling and provides denser packing of applications hosted by CSPs, while also facilitating dynamic sizing and live migration of containers such as Docker. Jelastic “simplifies” Private Cloud by providing an infrastructure that is “easy to deploy and simple to manage the entire enterprise stack and can install on ‘bare metal’ servers.”

Jelastic Simplified-Private-Cloud

 

Mirantis is the “number one pure-play OpenStack solutions provider, the most progressive, flexible, open distribution of OpenStack, combining the latest innovations from the open source community with the testing and reliability customers expect. More customers rely on Mirantis than any other company for the software, services, training and support needed for running OpenStack Cloud.” Mirantis is the only pure-play OpenStack contributor in the top five companies contributing open source software to OpenStack and supports many Global 1,000 companies such as AT&T, Cisco WebEx, Comcast, Dell, The Gap, NASA, NTT Docomo and PayPal to build and deploy production-grade OpenStack. Mirantis OpenStack is a “zero lock-in distro that makes deploying Cloud easier, more flexible and more reliable,” while OpenStack Express is an “on-demand Private-Cloud-as-a-Service” that allows users to deploy workloads “immediately.”

mirantis-openstack-essex-component-architecture-diagram-1-728

 

MuleSoft provides the most widely used integration platform for connecting any application, data source or API, whether in the Cloud or on-premises. With Anypoint™ Platform, MuleSoft delivers a complete integration experience built on proven open source technology, eliminating the pain and cost of point-to-point integration. Anypoint Platform includes CloudHub™ iPaaS, Mule ESB™ and a unified solution for API management, design and publishing. “CloudHub is the fastest way to integrate SaaS applications reliably and securely. The platform of choice for enterprise integration, CloudHub offers global availability and 99.99% up-time. Compliance with the highest security standards ensures integration are protected wherever they run.” MuleSoft has more than 170 enterprise customers.

mulesoft-application-architecture-the-next-wave-121029165105-phpapp02-thumbnail-4

 

Managed Cloud Services Providers

Many of the above-mentioned IaaS solution providers also offer Managed Cloud Services, including IBM and SunGard Availability Services. Telcos such as AT&T, NTT and Verizon also have viable offerings in this space. This space is also crowded with hundreds of service providers of every ilk. The primary reason for this is MCSPs make the transition to the Cloud much easier for most organizations.

The two examples below are MCSPs that leverage other provider networks/data centers and specialize – although not exclusively – within specific application areas (e.g. Avanade for Microsoft applications) or within specific industries (e.g. DataPipe in financial services).

Avanade “Private Cloud solution is built on the latest Microsoft technologies, including Windows Server 2012, System Center 2012 and Avanade’s own Cloud Services Manager software. These technologies enable IT to improve productivity, optimize operations, lower costs, tighten security and reduce your environmental footprint. Avanade also offers scalable, infrastructure-managed services that can help you rapidly cut costs. We offer a comprehensive selection of managed services including Service Desk coverage, Workplace Services, Data Center Services, Network Services and Security Services.” A joint venture between Accenture and Microsoft, Avanade seeks to accelerate the business value of Private Cloud with tailored managed services while providing clients with a flexible, agile business approach.

Avanade Cloud

 

Datapipe offers a single provider solution for managing and securing mission-critical IT services, including Cloud computing, infrastructure as a service, platform as a service, colocation and data centers. “Datapipe delivers those services from the world’s most influential technical and financial markets, including New York metro; Ashburn, VA; Silicon Valley; Iceland; London; Tel Aviv; Hong Kong; Shanghai and Singapore. Datapipe Managed Cloud for Amazon Web Services (MAWS) is a unique offering that combines the flexibility, scalability and power of Amazon’s world-leading Cloud platform with Datapipe’s award-winning support and managed services to power hybrid Cloud solutions with Datapipe High performance Oracle, SQL and MySQL database clusters delivered as a service over a direct connection between Amazon and Datapipe networks.” According to Netcraft, Datapipe’s impressive connect time, 16ms, is evidence of the benefits of their globally disperse hosting platform.” Datapipe offers a comprehensive suite of enterprise-grade managed security and compliance solutions.

Datapipe Managed Hosting

 

NoSQL Databases in the Cloud and Big Data

The popularity and viability of the Cloud coupled with the explosion of Big Data has engendered a variety of new databases that have broken the mold and, perhaps soon, the grip of traditional relational database management systems (RDBMS) – not to mention enabled new business models and initiatives that rely on fast access to ever-larger amounts of data – that have dominated enterprise IT shops for decades.

In the following graphic (Source: blog@tonyshan.com), Big Data stores, better known as databases, are arranged in three categories: SQL, NewSQL and NoSQL, which stands for Not Only SQL. The SQL space is dominated by IBM, Microsoft and Oracle traditional relational databases, whereas NewSQL and NoSQL have scores of newer entrants – including offerings from IBM (Cloudant) and Oracle (based on Berkeley DB).

Taxonomy of Big Data Stores_Tony Shan

Many of today’s more popular NoSQL DBs are offshoots or hybrids of DB initiatives developed by web-scale companies such as Amazon (Dynamo) and Google (Big Table) where traditional RDBMS were not suitable for “quickly” accessing large amounts of unstructured or semi-structured data such as documents, images, webpages, wikis, emails and other forms of “free” text.

As described in 21 NoSQL Innovators to Look for in 2020, there are at least four primary categories of NoSQL DBs, including Distributed, Document-Oriented, Graph DBs and In-Memory, with variances within each category. Many NoSQL DB solutions are based on open source solutions such as HBase, Cassandra, Redis or CouchDB along with Dynamo and Big Table.

Many of the almost 100 commercially available New or NoSQL DBs are optimized for the Cloud and offered by CSPs as DBaaS platforms for small companies or can be scaled out for very large private Cloud instances with multiple nodes and locations.

The following is a small sample of NoSQL DBs, including two of the most popular solutions and one relative start-up.

AtomicDB offers an “associative” DB approach that holds the promise of being able to manage exabytes of data far exceeding the capacity of conventional DB approaches or even Hadoop and HDFS. Developed for the Navy in the 90s for data warehousing applications, AtomicDB is now being commercialized for government, financial and healthcare use cases where the need to access exponential Big Data growth is paramount. “The underlying AtomicDB storage engine has patent pending technology for compression, encryption and obfuscation. The technology enables very high security and massive scalability to exabyte levels. This is possible with commodity hardware and Cloud platforms not requiring expensive proprietary appliances. Parallel processing and associative processing enables complex queries over large secure data sets up to two orders or magnitudes faster than alternatives.”

AtomicDB

 

MarkLogic is the market share leader for enterprise NoSQL implementations and now leverages AWS to offer clients additional resources to dynamically scale, “enabling the right resources to be brought to bear even during peak times. A combination of advanced performance monitoring, programmatic control of cluster size, sophisticated data re-balancing and granular control of resources bring agility and elasticity to the data layer, whether it is on-premise or in the Cloud.” With increased interest in accessing large data sets stored in the Cloud, MarkLogic is poised to deliver Semantic Technologies, “unleashing the power of documents, data and triples so you can understand, discover and make decisions based on all kinds of information available from many sources in a single, horizontally scalable database to handle XML, text, RDF, JSON and binaries, and to query across all your information – eliminating redundancy, complexity and latency.”

MarkLogic_Semantics_Slide

 

MongoDB has the largest user community and partner ecosystem in the NoSQL space due to its popular open source distribution. Its DBaaS is hosted by AWS and IBM/Softlayer – a big partner and supporter – Rackspace, Cumulogic and many others. “MongoDB stores data as documents in a binary representation called BSON (Binary JSON). MongoDB Management Service (MMS) is the application for managing MongoDB, created by the engineers who develop MongoDB. Using a simple yet sophisticated user interface, MMS makes it easy and reliable to run MongoDB at scale, providing the key capabilities you need to ensure a great experience for your customers, offering automated deployment and zero-downtime upgrades disaster recovery and continuous monitoring in the Cloud.”

MongoDB Cluster

 

Cloud (In)Security

Security concerns, followed by compliance and regulatory considerations, is typically the top reason enterprises give for not moving data or Tier 1 apps to the Cloud. As CSPs, PaaS and SaaS vendors point out, security solutions and policy enforcement are required at each layer and there is no one product or service provider that Parity can find today that addresses the entire stack and all of the potential gaps and vulnerabilities.

CSPs such as AWS insist their IaaS and PaaS layers are highly secure and are quick to point out that any breaches their clients have experienced were due to failures on the customer’s end, whether it be poor data encryption policies, network breaches or identity management issues. Moreover, there are few if any security policy standards for the Cloud industry for encryption or Common Criteria (NIST and ISO are exceptions but not complete frameworks for Cloud) across the entire stack as each CSP usually offers different levels of services – even within their own portfolio of products and services.

According to a senior IBM security executive Party interviewed, “Everyone needs to start over and rethink security architecture and controls, especially secondary and tertiary levels. The risks have changed and are changing more rapidly as time goes on. Therefore, auditable controls and security must be built in upfront, not as an afterthought.” As is the case with two other giants in the enterprise security space, RSA and Symantec, IBM is busy developing and acquiring security solutions to address gaps in Cloud security.

Very recently, IBM announced the acquisition of Lighthouse Cloud Security Services to address major Cloud security gaps in Identity and Access Management (IAM) and the growing need to address global IAM deployment and management challenges as well as combining Managed Security Services and IAM. At the same time, the AWS ecosystem is attracting a number of security point solutions to address the gaps and vulnerabilities that enterprises are confronted with on their end.

Despite the fact that every Cloud-enabled solution provider addresses security at some level in the Cloud stack, enterprises need to understand where their responsibilities lie – especially at the policy level.

The following is a short list of mostly smaller Cloud security vendors, all of whom provide security analytics, assessment and/or remediation capabilities.

Evident.io is a “fast, safe and simple solution for AWS security. The Evident Security Platform (ESP) performs continuous AWS security monitoring as a service and can identify and assist you in correcting problems in as little as 5 minutes. ESP for AWS identifies over 100 critical AWS security vulnerabilities across all of your AWS accounts. Security risks are color-coded and displayed on the ESP Risk Assessment Dashboard intuitively, facilitating rapid problem identification in minutes rather than days or weeks. Click on any security issue to get a comprehensive overview and expert instruction on how to remediate the problem as quickly as possible. The Evident Security Platform offers a cost-effective solution that dramatically reduces time spent finding and fixing AWS security risks, resulting in a significantly safer Amazon Cloud experience.”

Evident.io screenshot-2

 

Porticor offers a rich variety of Cloud encryption capabilities. “Your project’s needs and characteristics will determine the right choices for your application. The Porticor Virtual Private Data solution includes two or three major components: Porticor’s Virtual Key Management Service (PVKM) – a unique and patented key management technology provided as a service. PVKM is stronger than hardware, thanks to patented technologies such as Split-Key Encryption and Homomorphic Key Management; a Porticor Virtual Appliance (one or more for high availability), implemented inside your Cloud account; and an (optional) Porticor Encryption Agent, which may be installed and used on one or more of your Virtual Machines (your servers).” Porticor has Cloud encryption and key management solution for AWS and VMware and supports compliance requirements such as HIPAA. The diagram below represents an overview of the deployment options.

Porticor cloud-encryption-deployment-scenarios

 

Splunk provides operational intelligence apps for AWS Cloud that “ensure adherence to security and compliance standards and [allows users to] gain visibility into AWS billing and usage. Splunk Enterprise transforms machine data into real-time operational intelligence. It enables organizations to monitor, search, analyze, visualize and act on the massive streams of machine data generated by the websites, applications, servers, networks, mobile and other devices that power the business. Splunk Enterprise is available as a free download on your laptop; then deploy it to your data center or Cloud environment. This machine-generated data holds critical information on user behavior, security risks, capacity consumption, service levels, fraudulent activity, customer experience and much more. This is why it’s the fastest-growing, most complex and most valuable segment of Big Data.”

splunk

 

Analytics, Data Integration and Visualization Tools

Analytics and data integration capabilities are core offerings of virtually every Big Data vendor and CSP, whether those offerings are inherently homegrown and proprietary, based on open source solutions or a hybrid approach that provides a combination of tools and relies on partners with specialized capabilities. The hybrid approach is certainly in play, more or less, with all the major enterprise grade CSPs, including AWS, HP, IBM and Microsoft.

With the advent of the Internet of Things, Social Media, various forms of electronic messaging (e.g. documents, email, IM), streaming media and video, images, voice, web pages, wikis and other forms of unstructured or semi-structured data combined with structured data from traditional RDBMSs and core systems such as ERP (e.g. SAP and Oracle) or CRM (e.g. Salesforce.com) solutions, data sources have become much more diverse.

All of this data diversity as well as large volumes of Big Data pose challenges and opportunities for organizations of all sizes. The promise of the Cloud and the thousands of solution providers who create the “magic” is access to a vast ocean of information and potential insight, whether publicly available or contained in an organization’s own private “Data Lake.” This democratization of data access is achieved through the combined efforts of a diverse Cloud ecosystem that enables small companies or divisions of large organizations to quickly adopt IT services to meet new and evolving business requirements.

What follows is a small sampling of vendors that offer analytics, data integration and visualization tools to help users manage and make sense of large volumes of data in the Cloud:

Cloudera Enterprise allows “leading organizations to put their data at the center of their operations, to increase business visibility and reduce costs, while successfully managing risk and compliance requirements along with robust security, governance, data protection, and management that enterprises require.” According to senior execs at Cloudera, “Customers have been running Cloudera’s Hadoop distribution (HDFS) in mission-critical environments, 24/7 for the past 3 years, thus proving that Hadoop is enterprise ready.” Cloudera offers “one massively scalable platform to store any amount or type of data, in its original form, for as long as desired or required; integrated with your existing infrastructure and tools; and flexible to run a variety of enterprise workloads – including batch processing, interactive SQL, enterprise search and advanced analytics.” Cloudera is the primary contributor to Apache-licensed, open source Impala, “the industry’s leading massively parallel processing (MPP) SQL query engine that runs natively in Apache Hadoop.”

Cloudera diagram-edh-2014-620x361

 

Crawford Technologies transforms print stream data that contains vital customer information, providing additional source data for enterprise data warehouses and analytics solutions. Crawford’s suite of software solutions and services “enable clients to meet even the most rigorous demands for instantaneous access to information, irrespective of current, legacy or future standards in infrastructure or document output. High-value document solutions streamline, improve and manage enterprise content, document archive and production and advanced output management. Our Document Accessibility Services (DAS) produces alternate customer communications formats for blind and partially sighted customers, as well as those unable to read traditional print. Conversions include PDF/UA, Braille, large print, audio and e-text. Organizations can quickly and automatically transform monthly banking, credit card or investment statements to meet the ever-growing needs of an under-served portion of the population.” Crawford solutions extract customer-facing data to support financial services compliance requirements. Key partners include EMC and IBM.

Crawford Insurance example

 

Hortonworks is “architected, developed and built completely in the open. Hortonworks Data Platform (HDP) provides Hadoop designed to meet the needs of enterprise data processing. HDP is a platform for multi-workload data processing across an array of processing methods – from batch through interactive to real-time – all supported with solutions for governance, integration, security and operations. As the data operating system of Hadoop 2, Apache Hadoop YARN breaks down silos and enables you to process data simultaneously across batch, interactive and real-time methods. YARN has opened the Data Lake to rapid innovation by the ISV community and delivered differentiated insight to the enterprise. Increasingly, enterprises are looking for YARN-ready applications to maximize the value of its data. As key architects of YARN, we cherish open architecture and are committed to collaborate with ISVs and partners to onboard their applications to YARN and Hadoop.”

Hortonworks

LogiAnalytics’ mission is to bring a self-service, Google-like experience to non-programmers by providing a code-free development environment that includes “our unique ‘Elemental Design’ approach, which provides hundreds of pre-built modules to speed design and development of your BI applications. This provides increased agility and flexibility, freeing your organization from long implementation cycles and extensive re-coding. Our architecture makes it easy to build tailored applications to deliver views of any kind of information, customized to each user. Drive interactivity with employees, partners, customers or the public in a way that serves your business needs. The design of our modular architecture and flexible API allows enterprise business intelligence functions to be embedded directly inside of existing enterprise applications, delivering critical information where users can act on it, dramatically speeding both adoption and deployment. Our write-back functionality integrates BI completely within your operational infrastructure.” Major partners include HP Vertica and MongoDB.

LogiAnalytics Infused Analytics

 

Pentaho “data integration prepares and blends data to create a complete picture of your business that drives actionable insights. The complete data integration platform delivers accurate, ‘analytics-ready’ data to end-users from any source. With visual tools to eliminate coding and complexity, Pentaho puts Big Data and all data sources at the fingertips of business and IT users alike. Within a single platform, our solution provides visual Big Data analytics tools to extract, prepare and blend your data, plus the visualizations and analytics that will change the way you run your business. Pentaho’s flexible Cloud-ready platform is purpose-built for embedding into and integrating with your applications. Our powerful embedded analytics combined with flexible licensing and a strong partner program [e.g. Cloudera, MongoDB, HP Vertica, Hortonworks) ensures that you can get to market quickly, drive new revenue streams and delight your customers.”

Pentaho evolving-big-data-architectures

 

SmartLogic “content intelligence platform, Semaphore, uses semantics, text analytics and visualization software technologies to perform five primary functions: Ontology and taxonomy management; Auto-classification of unstructured data; Text analysis (including entity, fact and sentiment extraction); Metadata management; and Content visualization. The resulting metadata drives an array of business-critical tasks, including semantic user experience for search, text analytics, workflow processes driven by meaning, regulatory compliance involving unstructured content, automatic classification for content management, decision support using information locked-up in content. Semaphore classifies content using semantic models such as taxonomies and ontologies. It automatically generates metadata that represents each classification decision. Semaphore is tightly integrated with content management systems, business intelligence systems and enterprise search engines and offers interoperability with graph databases and triple stores.” Semaphore integrates with many existing systems and can be customized to work with enterprise solutions such as SharePoint and MarkLogic.

SmartLogic

 

Syncsort has effectively leveraged its legacy mainframe solutions business to provide mainframe users with the ability to extract Big Data for ingestion into Hadoop EDW environments. While mainframes are a good source of Big Data, mainframes do a lot of sorting at a comparatively high cost. “Syncsort maximizes Hadoop’s full potential by tapping into any data source and target, including mainframes. No coding, no scripting, just faster connectivity. Syncsort DMX is full-featured data integration software that helps organizations extract, transform and load more data in less time, with less money and fewer resources. DMX processes data 10x faster than other data integration solutions, easily scales to support growing data volumes, eliminates manual coding and performance tuning and lowers data integration TCO by up to 65%. Smarter Data Integration means faster data processing with less hardware and IT resources to harness the opportunities of Big Data.”

Syncsort DMX Architecture

 

Agile Application Transformation and Development Tools

Arguably very similar to several of the vendors listed in the prior Analytics, Data Integration and Visualization category, along with some elements of PaaS, what both examples listed below have in common is they allow organizations to migrate legacy applications, along with the associated business logic, to the Cloud. This class of solution offers a higher level of self-service capabilities for both IT and business users by limiting the amount of hand coding necessary, therefore shrinking application development and deployment cycles by several weeks.

Appian modern work platform combines the “best of business process management, social business, mobile access and Cloud deployment. Business Process Management Software in the Cloud enables strategic process improvement, reduced technology cost, and better alignment of IT with business goals. The new IT paradigm and business model can drive new growth opportunities, increase profit margins for the private sector and achieve more efficient and effective missions for federal agencies. The entire Appian BPM Suite is available as either an on-premise or Cloud offering. The Appian BPM Suite delivered in the Cloud is a secure, scalable and reliable way to create a more nimble and adaptive organization. The benefits of deploying Appian in the Cloud include low startup costs, fast deployment with no manual maintenance, predictable costs during the life of the application and fast return-on-investment.”

Appian Mphasis-Hybrid-Agile-Method

 

OutSystems is designed to appeal to the “citizen programmer” and provides the “only open, high-productivity application platform (PaaS) that makes it easy to create, deploy and manage enterprise-mobile and web applications – helping IT deliver innovative business solutions fast. OutSystems Platform enables rapid delivery of beautiful applications for all devices and empowers IT to attack changing business requirements.” The “High Productivity Platform” can run on a desktop, tablet or smart phone and is available as a public Cloud, private Cloud and on-premises solution. The OutSystems Platform allows programmers to “focus on building a product, not scaffolding, helping organizations get to market faster and better” by generating C# (Sharp), HTML, referential database code or any other code or artifacts necessary to make applications work while also tying data to existing data sources.

OutSystems

 

Cognitive Computing in the Cloud

This is yet another emerging class of solutions for building applications in the Cloud that take advantage of neural networks and other advances in artificial intelligence (AI) and cognitive computing technologies. Suffice to say that virtually all the major application development vendors, including HP (IDOL) and IBM (Watson), are using some form of cognitive computing. The three vendor examples here are all very small – which is where much of the innovation is expected to come from in the future.

Cortica is an intelligent visual search tool for the web founded in 2007 by a team of Technion researchers who “amazed by the computational capabilities of biological systems, reverse-engineered the Cortex to enable a new generation of biologically inspired computational technologies. Despite the major technological breakthroughs of recent decades, basic tasks such as image recognition, movement in the physical world and natural language understanding – easily performed by toddlers – are practically impossible for even the most advanced computers. The essence of Cortica’s Image2Text® technology lies in its ability to automatically extract the core concepts in photographs, illustrations and video and map these concepts to keywords and textual taxonomies. Images drive interest, sentiment and commercial intent, and Cortica’s technology reads and automatically associates images with relevant content in real time. This groundbreaking model gives our partners a completely new way to engage a highly targeted mass audience.” Cortica works with phones and wearables.

Cortica Image recognition

 

Ersatz Labs “aims to make deep neural network technology available to the masses. Ersatz is the first commercial platform that packages state-of-the-art, deep-learning algorithms with high-end GPU number crunchers for a highly performant machine-learning environment. We sell it in the Cloud or in a box. Ersatz is the easiest way for anyone to start building applications that take advantage of deep neural networks. With Ersatz, you can focus less on feature engineering and more on collecting as much data as you can, even if it is unlabeled. One of the most powerful benefits of deep neural networks is their apparent ability to extract powerful features from your data automatically. The formatting of your data may be task specific, but in general, the following tasks are supported: classification, clustering, feature extraction, dimensionality reduction, time series prediction, regression, and sample generation.”

ersatz_screen_data

 

Narrative Science introduced Narrative Analytics, “a new approach to automated communication that starts with the story. The story drives a set of communication goals that reflect what needs to be said. These, in turn, define the analysis that needs to be performed in order to get the facts that support the story. The result is a complete set of requirements to drive the creation of the narrative. This linkage between story, analytics and data – the core tenet of Narrative Analytics – is fundamental in making sure you say the right thing at the right time. Quill is a patented AI platform that goes beyond merely reporting the numbers – it gives you true insight on your data. Quill uncovers the key facts and interesting insights in the data and transforms them into natural language produced at a scale, speed and quality only possible with automation.”

Narrative Science What_Is_Quill_R7b

 

End Note:  For a peek into how large Banks and Financials are looking to collaborate on Cloud de facto standards, reference architectures and taxonomies click on the following link. IaaS and PaaS: Mature Enough for Financial Services Firms?

 

About Gary MacFadden

Gary's career in the IT industry spans more than 25 years starting in software sales and marketing for IBM partners DAPREX and Ross Data Systems, then moving to the IT Advisory and Consulting industry with META Group, Giga and IDC. He is a co-founder of The Robert Frances Group where his responsibilities have included business development, sales, marketing, senior management and research roles. For the past several years, Gary has been a passionate advocate for the use of analytics and information governance solutions to transform business and customer outcomes, and regularly consults with both end-user and vendor organizations who serve the banking, healthcare, insurance, high tech and utilities industries. Gary is also a frequent contributor to the Wikibon.org research portal, a sought after speaker for industry events and blogs frequently on Healthcare IT (HIT) topics.
This entry was posted in Big Data, Cloud Computing, Information Management Thought Leadership and tagged , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , . Bookmark the permalink.

Leave a Reply