News

January 21, 2019 21 Jan'19
Data.gov shutdown shows limits of open data

The Data.gov shutdown shows that, as open data can be turned off, data professionals may need to consider alternative sources for the kinds of data the government offers.
January 15, 2019 15 Jan'19
Cloudera and Hortonworks combo to push CDP, machine learning

Two wunderkinds of Hadoop have formalized their merger. Cloudera and Hortonworks say they will place special focus on AI as they chart the stand-alone vendor's future.
January 10, 2019 10 Jan'19
IBM Weather Channel location data use spurs privacy concerns

IBM CEO Ginni Rometty pushed real-time weather data mining for travel and other uses, even as the Los Angeles city attorney is suing IBM's Weather Company unit over its sharing of location data with business partners.
December 28, 2018 28 Dec'18
Data management trends for 2019: Governance, DataOps, cloud

Better data governance, increased cloud use and wider DataOps adoption head the list of trends for data management teams to plan for in 2019, IT analysts say.
December 18, 2018 18 Dec'18
Datawatch Swarm on path to combine data prep, analytics

Datawatch plans to add analytics and data visualization tools to its Swarm data preparation platform, starting with integration capabilities in a Swarm 2.2 release.
Sponsored News
- How IT leaders can prepare IT architectures for an AI future using Intel Xeon® with DL Boost
  
  Sponsored by Intel - IT leaders are preparing their organisations for increasing use of artificial intelligence (AI) to extract the insights they need from growing amounts of data. To help them, Intel is enabling businesses to future-proof their IT architecture by building AI accelerators into its second-generation Xeon® scalable processors. See More
- Delivering datacentre performance and cost savings with Intel Optane technology
  
  Sponsored by Intel - Intel® Optane™ technology is the first memory and storage breakthrough in 25 years. Based on Intel® 3D XPoint™ memory media, it offers a new generation of solid-state devices (SSDs) and memory modules built from the ground up. See More
- Improve Datacentre Performance by Closing the Gaps in Memory and Storage Hierarchy
  
  Sponsored by Intel - Remember what it felt like when you got your first solid state drive (SSD)? Back in 2008, laptops with a hard disk drive (HDD) felt unresponsive – they lagged and nearly ground to a halt when the virus scanner started accessing the HDD. Then Intel launched the X-25M SSD and laptops felt much more responsive. Gone was the lag. Nobody noticed when the virus scanner ran. See More
- Where to start with AI – the journey from business need to delivery
  
  Sponsored by Intel - Businesses are excited about artificial intelligence (AI) and the benefits it offers, but like so many new technologies, that potential can be wasted if you don’t know where to begin the journey to production-ready AI. See More
View All Sponsored News
December 14, 2018 14 Dec'18
Data companies vie to fill spots in AWS cloud data lineup

Third-party vendors that offer data platforms to AWS users tout hedges against cloud lock-in. But they must both compete and collaborate with the cloud leader.
November 30, 2018 30 Nov'18
Amazon Timestream, QLDB aim to grab more DBMS users for AWS

Built-for-purpose databases target general ones, especially in the cloud. At re:Invent 2018, AWS added time-series and transaction ledger databases to expand its line.
November 27, 2018 27 Nov'18
Microsoft Azure SQL Database and Oracle 19c start to rev

A new Microsoft Azure SQL Database Managed Instance service seeks to span the gap between cloud and on-premises systems. Meanwhile, Oracle also has cloud plans of the data kind.
November 06, 2018 06 Nov'18
IBM-Red Hat deal highlights changes due in data processing

The IBM acquisition of Red Hat marks a watershed in computer architecture. The duo says it can rebuild data applications in new ways. This news analysis explores what's coming.
October 12, 2018 12 Oct'18
MarkLogic Data Hub Service aims to ease cloud use of NoSQL DBMS

MarkLogic rolled out a cloud-service version of its NoSQL database management system, a move designed to make the technology more cost-effective for cloud users.
October 04, 2018 04 Oct'18
Cloudera-Hortonworks merger narrows Hadoop users' options

Hadoop users will have fewer choices as big data rivals Cloudera and Hortonworks unite. But the new company may be more competitive with AWS and Google.
September 26, 2018 26 Sep'18
Mining equipment maker uses BI on Hadoop to dig for data

BI on Hadoop is still new, but moving BI to data is trending. A data scientist working with IoT data at Komatsu sees the importance of getting big data to the right people.
September 13, 2018 13 Sep'18
Containers key for Hortonworks alliance on big data hybrid

Hortonworks is joining with Red Hat and IBM to work together on a hybrid big data architecture format that will run using containers both in the cloud and on premises.
September 07, 2018 07 Sep'18
Big data tooling rolls with the changing seas of analytics

Hadoop data tooling is expanding. A view holds that Hadoop is moving from alternate data warehousing to a full-fledged big data analytics offering.
September 04, 2018 04 Sep'18
Graph processing gives credit analysis firms an edge

A graph database startup's parallel loading, processing and querying combine to deliver real-time data for fintech firms that offer fast credit evaluations online.
August 08, 2018 08 Aug'18
Confluent Platform 5.0 aims to mainstream Kafka streaming

Confluent Platform updates seek to bring data streaming with Apache Kafka to a wider audience. A new GUI and user-defined functions are part of the 5.0 release.
July 19, 2018 19 Jul'18
Focus, scope and spotting opportunity are key to role of CDO

Chief data officers and experts see the CDO role as changing to a more strategic orientation -- especially finding key opportunities in vast troves of data.
July 16, 2018 16 Jul'18
Chief data officer role: Searching for consensus

The chief data officer role is about many things -- regulations, innovation, AI and more. Consultant Randy Bean discussed the matter ahead of an MIT symposium on the topic.
July 02, 2018 02 Jul'18
Hadoop data governance services surface in wake of GDPR

GDPR influence is touching a Hadoop big data world that was immune to many privacy considerations until now. This podcast features the rise of Hadoop data governance for data lakes.
June 27, 2018 27 Jun'18
MongoDB 4.0, Stitch aim to grow NoSQL apps in cloud, on-prem

NoSQL vendor MongoDB upgraded its database software with ACID support, while also releasing a serverless platform intended to simplify application development.
June 25, 2018 25 Jun'18
Hadoop data lake architecture tests IT on data integration

Hortonworks users talk about building Hadoop data lakes to support new applications -- and the challenges their teams face on ingesting and refining data for end users.
June 18, 2018 18 Jun'18
Hortonworks cloud options grow via Google, Microsoft, IBM

Hortonworks now supports Google Cloud Storage and has also broadened cloud deals with Microsoft and IBM, aiming to increase cloud uses of its big data platform.
March 21, 2018 21 Mar'18
Kubernetes gains momentum in big data implementation process

Big data vendors and users are looking to Kubernetes-managed containers to help accelerate deployments and enable more flexible use of computing resources.
March 15, 2018 15 Mar'18
Streaming tool from StreamSets eyes data in motion for GDPR

StreamSets software for inspecting big data brings governance to data in motion. Such capabilities may find more use as the European Union's GDPR deadline looms.
February 22, 2018 22 Feb'18
Google TPUs open up on cloud; LinkedIn intros Hadoop Dynamometer

In big data news, we find Google TPUs, or Tensor Processing Units, offered as a cloud service, while LinkedIn is open sourcing a Hadoop test simulator called Dynamometer.
February 16, 2018 16 Feb'18
MongoDB 4.0 takes ACID transactions to multi-document level

MongoDB is taking a deeper step into SQL-style processing waters with a 4.0 update that brings increased support for ACID-compliant transactions to its NoSQL database.
January 03, 2018 03 Jan'18
Apache Hadoop 3.0 goes GA, adds hooks for cloud and GPUs

Is this the post-Hadoop era? Not in the eyes of Hadoop 3.0 backers, who see the latest update to the big data framework succeeding in machine learning applications and cloud systems.
December 01, 2017 01 Dec'17
AWS graph database Amazon Neptune sets sail at re:Invent 2017

After Cyber Monday, Amazon Web Services techies headed for re:Invent 2017. There, among a deluge of cloud product announcements by AWS, the Amazon Neptune graph database surfaced.
December 01, 2017 01 Dec'17
Apache Impala gets top-level status as open source Hadoop tool

Born at Cloudera, the MPP query engine known as Apache Impala has become a top-level open source project. It's one of various tools bringing SQL-style interactivity to big data analytics.
October 30, 2017 30 Oct'17
Connectedness is king, as Neo4j graph database ports to Spark

The Neo4j graph database emphasizes easy relationship mapping for diverse data points. Now, its related Cypher query language is hooking into Apache Spark.
October 05, 2017 05 Oct'17
Big data systems up ante on data quality measures for users

At the Strata conference in New York, IT managers detailed steps they're taking to improve data quality in their big data environments in order to help ensure analytics accuracy.
September 28, 2017 28 Sep'17
MapR-DB uses NoSQL to drive real-time analytics

The Strata conference in New York saw big data platform vendor MapR Technologies update MapR-DB, its NoSQL database engine, to better perform in real-time analytics applications.
September 26, 2017 26 Sep'17
Bank sees ETL processes evolve from IT to front-line users

ETL jobs -- once the sole province of IT -- take on a new form as data wrangling and self-service gain greater traction with business users of analytics.
September 06, 2017 06 Sep'17
Energy partnership drills down on data quality issues

Breitburn Energy Partners employed data quality tools to address the business pain of bad data, using the software to give end users the means to fix data quality issues themselves.
August 31, 2017 31 Aug'17
Confluent's Kafka data-streaming framework gets 'SQL-ized'

SQL on Hadoop arrived -- so did SQL on Spark. Now, SQL on Kafka is emerging to provide a different way to look at Kafka data as it streams through the enterprise.
August 30, 2017 30 Aug'17
Big data vendors, users see Hadoop stack from different sides

In this Talking Data podcast, TechTarget editors discuss Hadoop's future, IBM's decision to resell the Hortonworks distribution of the open source technology and other big data issues.
July 31, 2017 31 Jul'17
Apache Arrow to power startup Dremio's self-service data platform

Data management startup Dremio has aimed its Apache Arrow expertise at the problem of self-service data delivery. In-column caches and optimization speed queries across varied data stores.
July 24, 2017 24 Jul'17
Chief data officer jobs call for nurturing data ethos in companies

In many organizations, chief data officer jobs centered on defense against risk are giving way to ones emphasizing innovation. To do so, CDOs must nurture a data culture, MIT panelists said.
July 10, 2017 10 Jul'17
MongoDB 'Charts' a way to dashboard visualization from its database

MongoDB targets better dashboard visualization with MongoDB Charts, which adds another means for business users trying to look into their NoSQL data pools.
June 30, 2017 30 Jun'17
Agile database development spurs changes in data tech, DBA roles

The quest for the agile database is putting developers in the forefront and has some DBA tasks moving to the development groups, according to panelists at a conference in Boston.
June 29, 2017 29 Jun'17
Data governance crucial to comply with EU's GDPR legislation

With the EU's new General Data Protection Regulation looming on the horizon, companies -- including many in the U.S. -- need to get going on required data governance upgrades.
June 23, 2017 23 Jun'17
MongoDB Atlas footprint grows bigger with Azure, Google cloud support

MongoDB has expanded cloud coverage for its Atlas hosted database service, with Azure and Google versions joining an initial AWS-based offering to give users a choice on cloud platforms.
June 20, 2017 20 Jun'17
IBM bets on Hortonworks Hadoop platform as its big data engine

IBM pulled the plug on its distribution of Hadoop in favor of reselling Hortonworks' bundle of big data technologies, a decision that reduces the number of Hadoop vendors to four.
May 31, 2017 31 May'17
Data prep for deep learning applications means careful planning

Deep learning applications often require a mix of data, and assorted preprocessing techniques. That makes data preparation a priority, and conventional machine learning may have a role to play.
May 12, 2017 12 May'17
Startup Confluent makes Kafka cloud play to ease streaming deployment

Kafka is a linchpin in many on-premises big data pipelines. Now, software vendor Confluent is offering a Kafka cloud service to ease use of the messaging and data streaming system in the cloud.
April 28, 2017 28 Apr'17
Building a data lake takes meticulous planning -- and flexibility

Data lakes offer a more expansive alternative to data warehouses for analytics uses. TDWI analyst Philip Russom offers advice on how to get things right in a data lake architecture.
April 28, 2017 28 Apr'17
Systems of record take back seat to systems of engagement, innovation

Systems of engagement represent a hotbed of activity in data management these days. Flexibility and scalability are watchwords.
April 20, 2017 20 Apr'17
Spark processing engine more at home in cloud, Databricks CEO says

Corporate users are becoming more open to deploying big data systems with Apache Spark in the cloud, Databricks CEO Ali Ghodsi says in a Q&A on the open source processing platform.
April 14, 2017 14 Apr'17
Persistence pays off for software containers in big data

Software containers encapsulate complexity and ease deployment, two traits that are helping to elicit growing interest in using them as part of big data systems.
March 31, 2017 31 Mar'17
AWS data lake pumps up Beachbody's analytics architecture

Fitness company Beachbody set up a data lake system in the AWS cloud to support big data analytics applications after deciding that an on-premises deployment would be too complicated.
March 24, 2017 24 Mar'17
Blockchain data disruption aborning, IDC analyst says

Blockchain data technology disruption may be in the offing. IDC's Stewart Bond says architecture at the core of controversial bitcoin may show a new path to data integrity.
March 10, 2017 10 Mar'17
Finance data governance program gets new start, broader focus

Businesses constantly need to evolve their programs for governing data. Nationwide's finance data governance team shares how it stepped up data governance strategy and processes.
March 08, 2017 08 Mar'17
Pepperdata calls on Dr. Elephant for big data application profiling

Application profiling software from Pepperdata is built on LinkedIn's Dr. Elephant open source entry. A primary goal is to get more Hadoop and Spark applications into production.
February 21, 2017 21 Feb'17
Containers help move Spark and Hadoop into analytics operations

Moving custom Spark and Hadoop pilot projects into production use has proved daunting. But container technology eased the transition at the Advisory Board analytics service.
February 16, 2017 16 Feb'17
Drizzle on tap to spur Spark Streaming architecture

Spark Streaming architecture to date has focused much on programming perks. Now, as a bit of a hedge against other streaming choices, Drizzle comes to bat to cut streaming latency.
February 06, 2017 06 Feb'17
Preparing data for analysis helps JTV find predictive gold

Predictive models help Jewelry Television's on-air hosts sell its wares, thanks to data integration and preparation processes that funnel a mix of data into the analytics applications.
February 06, 2017 06 Feb'17
Data preparation steps increase to meet predictive analytics needs

Data scientists building predictive models and machine learning algorithms often have to do more data preparation work upfront than is necessary in conventional analytics applications.
February 06, 2017 06 Feb'17
Fast-moving big data changes data preparation process for analytics

Increased automation of data pipelines and more flexibility for data scientists through self-service software are taking hold as big data deployments change data preparation practices.
February 03, 2017 03 Feb'17
Kaiser's data governance model leans heavily on data stewards

The head of Kaiser Permanente's data governance program says data stewards hold the key to the initiative's success, and he offers advice on managing data stewardship processes.
January 31, 2017 31 Jan'17
Data governance process gets fine-tuned, as big data makes its mark

Big data analytics and digital transformation challenge the conventional data governance process, with many questions to answer in organizations. But a governed data lake shows how it can be done.
December 02, 2016 02 Dec'16
Amazon ushers in pay-by-query pricing with Athena data engine

Amazon's Athena data engine brings interactive SQL queries to S3 data sets and lets users pay as they go. It's based on an open source framework called Presto that Teradata and others also employ.
November 30, 2016 30 Nov'16
Louisiana uses GIS software to create map to flood preparedness

The Louisiana Department of Health responded to flooding with the help of GIS software that located trouble spots with at-risk hospitals. Ease of use was welcome, according to a preparedness manager.
October 28, 2016 28 Oct'16
Containers and microservices find home in Hadoop ecosystem

Big data is moving from its bare-metal roots, and data streaming is a driver. Containers and microservices may have a role to play in what's next. An e-commerce application shows the way.
October 06, 2016 06 Oct'16
Effective data visualization crystallizes a company's crystal ball

What's in your toolbox? October's issue of Business Information turns the tables and puts that burning question to Capital One and several other business intelligence and data analytics software users. As the burgeoning worlds of ...
September 30, 2016 30 Sep'16
Vendors move to make Hadoop cloud deployments simpler, less costly

Users increasingly are eyeing the cloud for big data management and analytics applications, and IT vendors are moving to ease the process -- and the price -- of running Hadoop in the cloud.
September 29, 2016 29 Sep'16
MemSQL 5.5 adds native data ingestion for Kafka, eases pipelining

Among a handful of new SQL-oriented, in-memory databases is MemSQL. Recent product updates are meant to improve data pipeline creation and performance in high-speed ingestion applications.
September 15, 2016 15 Sep'16
HPE sells Vertica analytics, thanks to the growth of open source software

HPE is paring down its software holdings, including analytical database software in the Vertica line and other big data tools. A sale to Micro Focus is due to close next year, leaving users in some limbo for now.
August 31, 2016 31 Aug'16
HPE adapts Vertica analytical database to world with Hadoop, Spark

Vertica 8.0 expands the analytical database's support for Kafka, Spark and Hadoop. That's an important step, as the Hewlett Packard Enterprise technology tries to compete in a field of diverse data tools.
August 30, 2016 30 Aug'16
Snowflake cloud data warehouse targets gaps big clouds overlook

Cloud data warehouse offerings from smaller vendors seek to address functionality gaps that bigger players may miss. Newcomer Snowflake Computing targets concurrent queries, for example.
August 23, 2016 23 Aug'16
Cloud-managed database services change data administration

Managed data services are growing in use, as types of data stores proliferate and the cloud becomes the home for more data. DevOps is a driver behind the changes, which bring new duties and needed skills for DBAs.
July 22, 2016 22 Jul'16
The chief data officer's dilemma: CDO role in flux

How to balance data safety with innovative big data expansion was at issue at an MIT symposium where the chief data officer role was considered.
July 15, 2016 15 Jul'16
Chief data officer job stakes claim in data innovation

Forces at work in data management have led to the advent of the chief data officer. The role of the CDO and more is discussed in a Q&A with consultant Joe Caserta.
July 08, 2016 08 Jul'16
Flexible MongoDB connects to Spark analytics, adds DBaaS

Speed to production underlies interest in MongoDB. At its annual confab, the company behind the NoSQL database rolled out improvements, including a Spark analytics link.
July 06, 2016 06 Jul'16
IT teams hone Hadoop monitoring, governance to boost big data value

Hadoop management is becoming a bigger priority for big data users and vendors alike as the distributed processing framework plays a more central role in the business operations of organizations.
June 29, 2016 29 Jun'16
As field data gets fresher, it drives real-time decision making

Thanks to ubiquitous mobile technology, field data is more readily accessible to ESRI's Survey123 for ArcGIS. A relief organization used the software to aid Syrian war refugees.
June 24, 2016 24 Jun'16
DataStax gets graph DB and Titan gets rewrite

DataStax Enterprise 5.0 couples a Cassandra column-family data store with a rewritten version of the open source Titan graph database. The goal is fast analytics closely tied to fast transactions.
June 24, 2016 24 Jun'16
Fifty years forward: A look at the ComputerWeekly back pages

On the occasion of ComputerWeekly's 50th anniversary, Brian McKenna joins the Talking Data podcast crew to look back at Bletchley Park, and forward to Hadoop and AI.
June 17, 2016 17 Jun'16
Graph technology the beating heart of new data management tools

Graph technology is popping up in many places, including master data management. A major data integration player has joined the quest, as seen recently at Informatica World.
June 09, 2016 09 Jun'16
Databricks reveals more on Spark 2.0; IBM, Microsoft renew Spark push

Spark 2.0, with structured streaming and SQL 2003 support, is aborning as indicated at Databricks' Spark Summit, where R-to-Spark interfaces also popped up.
June 03, 2016 03 Jun'16
Kafka streaming gets a new twist

Startup vendor Confluent is looking to place a stake in the big data ecosystem with Kafka streaming and management tools meant to reduce complexity in applications that place data in motion.
June 01, 2016 01 Jun'16
MIT panel shares tips on how to get into the data business

Data wrapping -- in this case, bundling data and analytics services with products -- may entice more companies to become data businesses. A panel at an MIT symposium considered some best practices for doing so.
May 19, 2016 19 May'16
Oracle NoSQL: An oxymoron waiting to happen?

New cloud apps seem ready-made for NoSQL. This may cause Oracle to put more focus on its Oracle NoSQL database, which is often overlooked amid a crush of NoSQL contenders.
May 16, 2016 16 May'16
New data landscape augurs discovery-based architectures

In an interview, consultant Lakshmi Randall foresees changes in how data management is organized and executed as the overall data landscape shifts due to the adoption of big data systems.
April 29, 2016 29 Apr'16
Big data challenges traditional data modeling techniques

Surging big data is changing data modeling techniques, including schema creation. The word from Enterprise Data World 2016: Data pros must adjust.
April 29, 2016 29 Apr'16
EBay helps drive new style of data engineering

Open source data engineering has become a way of life at e-commerce leader eBay, says the company's Debashis Saha. Kylin is one of the tools that has resulted.
April 22, 2016 22 Apr'16
Data lake meets warehouse in hybrid data architectures

A new view on hybrid data architectures, in which data lakes and warehouses coexist, emerged at EDW 2016. The hybrid approach has implications for data design, skills and planning.
April 19, 2016 19 Apr'16
New tools offer a better view into managing Hadoop clusters

Running a Hadoop cluster in the data center isn't for the weak. But several new tools aim to give IT operations teams a closer look into what's going on inside Hadoop-based big data systems.
April 13, 2016 13 Apr'16
Hadoop market consolidation continues with Pivotal's exit

Pivotal Software dropped out of the Hadoop distribution business in favor of reselling the Hortonworks version of the big data framework -- and the market consolidation moves may not be over.
April 01, 2016 01 Apr'16
Streaming analytics puts data in motion at Strata + Hadoop 2016

Moving streams of data is a must in many modern applications. As a result, streaming analytics systems with Spark Streaming, Kafka and other components are coming to the big data forefront.
March 31, 2016 31 Mar'16
Hadoop core components may not be central to big data future

At Strata + Hadoop World 2016, Hadoop co-creator Doug Cutting said the core of the distributed processing framework is likely to see its position at the center of big data systems diminish.
March 25, 2016 25 Mar'16
Unstructured data is a misnomer

Nowadays, the term unstructured data pops up everywhere. It owes its popularity for a large part to the success of big data, to successful technologies such as NoSQL and Hadoop, and to formats such ...
March 24, 2016 24 Mar'16
Strata + Hadoop World 2016: Hadoop and Spark in spotlight

The Strata + Hadoop World conference focuses on big data management and analytics technologies, in particular the Hadoop distributed processing framework and Spark processing engine.
March 16, 2016 16 Mar'16
Social media startup uses NoSQL Redis Cloud to 'scale to infinity'

Because of growing data demands, and the need to nimbly scale up and down, a startup social networking platform chose a Redis Labs NoSQL database management system running on AWS.
March 02, 2016 02 Mar'16
Hortonworks Hadoop distribution goes to two release tracks

Looking to better balance system stability and innovation, Hadoop distribution provider Hortonworks will follow two release 'cadences' for different component sets in its HDP package.
February 29, 2016 29 Feb'16
Apache Spark architecture speeds data jobs, ousts MapReduce

Its collection of big-data processing features is priming the Apache Spark architecture for wider deployment. One key trait: Spark performance outpaces MapReduce in many Hadoop use cases.
February 24, 2016 24 Feb'16
AtScale Benchmarks Three SQL-on-Hadoop Engines

Numerous SQL-on-Hadoop engines are available for accessing data stored in HDFS using the familiar SQL language. They all look promising, they all support a rich SQL dialect, but which ones is the ...
February 24, 2016 24 Feb'16
Spark Streaming update to address growing torrent of big data

Amid the buzz at Spark Summit East 2016 in New York was word that the Spark data processing engine's stream processing architecture will be overhauled in the upcoming version 2.0 of the open source software.
February 04, 2016 04 Feb'16
HR metrics and analytics bring opportunity, concerns

HR personnel oversee many of the perks that come with a job, such as paychecks and benefits; the elements that most people would rather avoid, such as firings and employee conflicts; and all things in between, such as education and training. Now, ...
February 04, 2016 04 Feb'16
SQL-on-Hadoop platforms open brave new analytics world

Hadoop has been slowly plodding through the big data jungle, but SQL's integration may put a spring in the elephant's step.
February 03, 2016 03 Feb'16
NoSQL revs up to the tune of the Spark connector

Attention's been placed on Spark running on Hadoop, but there are Spark connectors for NoSQL that usher in a new class of operational analytics.

News

Sponsored News