In this article, I will outline why I use D3 and Dimple in SharePoint and a little about how to use it.
To be specific, my experience comes from developing on SharePoint 2013 on-premise installations, but with the use of AJAX and REST APIs for data collection, the technology can extend to other versions of SharePoint including SharePoint Online.
The article will cover four main points:
1. How I got hooked on D3 and Dimple.
2. What are D3 and Dimple?
3. Useful Characteristics of D3 and Dimple for SharePoint Development.
4. A Quick Overview of How to use D3 and Dimple in SharePoint.
Keeping things in perspective for SharePoint development: There is no one “Magic Bullet” technology. There is a host of components that come together to display a SharePoint page. Just to name the biggest players: There is HTML, CSS, JavaScript, jQuery, JSOM, CSOM, and C #. The list doesn’t even cover other languages for SharePoint App development.
D3 and Dimple are simply two more tools that can help in applications containing charts, graphs, and dashboards that might fill a niche for unique visualizations.
How I Got Hooked on D3.
And can I kick it? .
While working on some charting and graphing examples specifically with HighCharts (http://www.highcharts.com), a colleague introduced me to a fascinating D3 example:
https://bl.ocks.org/mbostock/4063269
The example shown to me was a D3 Bubble Visualization created by Mike Bostock from his d3 development hub on https://bl.ocks.org:
The visualization intrigued me and launched me into D3 exploration. I was at a loss of where to start. My background was in classic report writing.
As a programmer, I routinely separated data from graphics. My report writing occurred mostly in a Microsoft environment with SQL Server Reporting Services© (SSRS), Crystal Reports©, and ActiveReports©).
A very high-level view of my procedure for developing reports goes as follows:
1. Get the data source and connect to it.
Acquiring the data was usually the most expensive and intensive part of the operation in time and energy. The data source might be a SQL Stored Procedure, an Entity Framework or ADO.net Provider, a Web Service, an ODBC source, a CSV file, or maybe just the aggregation of some spreadsheets.
2. Once a data source is connected, there is a small amount of work arranging fields or columns on a grid or a matrix. There might be a little additional work formatting the headings, footers, and page numbers.
There was typically little investment of time and energy in the graphical output. The software generated the graphics from a limited number of templates. You simply chose the one to meet the reporting requirements.
An overly simplistic summarization would be: you hook up the data hose and let her rip.
The graphics I began to see with D3 required a real paradigm shift. As the graphics become more unique, a real investment in graphical presentation is required. This shift leads us to what D3 and Dimple are and how they work?
What are D3 and Dimple?
D3 stands for Data Driven Documents. The main website is D3js.org. D3 is a JavaScript library for producing dynamic, interactive data visualizations in web browsers. It makes use of the widely implemented SVG, HTML5, and CSS standards. It is the successor to the earlier Protovis framework. Mike Bostock is the undisputed developer and master of the technology.
D3 provides the engine to utilize underlying Scalable Vector Graphics (SVG)--which draws the visualizations. D3 is not the graphics layer but provides the Application Program Interface (API) to handle the XML Markup provided by SVG. Two excellent references on this relation are “Practical D3.js” and “D3.j s in Action.”
“D3 allows you to bind arbitrary data to a Document Object Model (DOM), and then apply data-driven transformations to the document. For example, you can use D3 to generate an HTML table from an array of numbers. Or, use the same data to create an interactive SVG bar chart with smooth transitions and interaction.” D3JS.org |
Just to get your feet wet with D3, Google “D3 1000 examples.” You will find there is even a site with 2000 examples. How often as a programmer do you get 1000 to 2000 hits from the “get go.” Many of these examples have detailed script with documentation and tutorials (I could retire now and spend the next 30 years plowing through tutorials, documentation, and examples. I don’t have anything planned past 96 anyway).
My favorite site for examples come from the gallery of Christopher Viau (http://christopheviau.com/d3list/gallery.html). Christopher is a graphical historian of D3 technology. He maintains a list of the top D3 resources in a visually attractive display which looks a lot like Mike Bostock’s http://bl.ocks.org but is more extensive in its coverage:
When you have a free moment just breathe in the panorama and scope of D3 technology offered by Christopher’s gallery
D3 technology is ubiquitous and yet stealthy at the same time. You might not have even been aware of its existence before reading this article, but you can’t escape it on popular websites.
Just look at the article “15 Data Visualizations That Will Blow Your Mind” to appreciate the penetration of D3 into the mainstream Web architecture.
D3 charts and graphs have appeared in numerous web articles with interactive graphics. If you follow sports like basketball, you’ve interacted with it in Bracket Selection displays. If you follow elections, there are interactive displays of what happens if a presidential candidate wins a specific state. Also, Word Graphs and detailed maps highlighting population or disease outbreaks are all generated from D3 programs.
D3 is not merely a charting and graphing software; it is a Data Visualization Engine.
As you might guess, D3 lends itself nicely to interactive client-side code like JavaScript and jQuery from which it derives.
Dimple simplifies D3.
“The aim of dimple is to open up the power and flexibility of d3 to analysts. It aims to give a gentle learning curve and minimal code to achieve something productive. It also exposes the d3 objects so you can pick them up and run to create some really cool stuff.” -Dimplejs.org
Take an example of a vertical bar chart. Natively, the bars would go down from the top. Think of it being negative y direction on an x/y-axis. There are a lot of transformations, formulas, and adjusting of margins to make the bar graph appear as everyone expects with the bar going upward. Dimple understands this and produces the chart as expected with less coding.
Please understand that Dimple doesn’t work without D3, and D3 doesn’t work without jQuery, and jQuery is toast without JavaScript enabled.
All this is nice but can it be used in SharePoint Programming? I’m glad you asked :)!
Useful Characteristics of D3 for SharePoint Development
· First and foremost, the technology is FREE for the taking. No third-party fees or restrictive licensing.
· Second, as already mentioned, there are tons of examples on the net with documentation and tutorials.
· Third, D3 has a variety of visualizations.
Why be limited. With a command of D3, a developer can morph displays from one visualization to another. The bar graph you are presenting can be sliced and diced by clicking on the legend and with a button click can change to a pie chart.
· Fourth, the library derives from jQuery.
The jQuery roots of D3 gave me a toehold on the technology. I was intimately aware the jQuery syntax of chaining and selecting. D3 was quite similar in some of its commands.
· Fifth, similar to jQuery, D3 has a command of HTML and Document Object Model (DOM). D3 can structure dropdowns, tables, and complex div structures. D3 can build the entire HTML page if needed.
I find building structures in D3 easier that jQuery but other programmers find another language easier--different tools for different programmers. I have found with D3 I can template different tables to display data from multiple views or lists using REST APIs and AJAX.
· Sixth, D3’s ability to bind to individual data elements allows the creation of amazing tooltips to describe a given element or provide a platform to even launch another application or widget.
· Seventh, D3 has a very good Date and Time functionality.
A project that I just finished building was a Monthly Calendar Widget that I use to display attachments from another list. The user can simply navigate to the month of interest and click on the attachment links for a specific meeting. The smart for the calendar came from D3.
The next section deals with the practicalities of using this technology in SharePoint, specifically SharePoint 2013 on premises.
A Quick Overview of How to use D3 in SharePoint
Nothing is ever easy to accomplish on a SharePoint Page. D3 and Dimple are just two more tools in the arsenal.
As a developer, I have been steadily avoiding doing anything with SharePoint WSPs for a number reasons. The first reason being server-side code is discouraged. Also, on large SharePoint sites, WSPs are hard to deploy and cause interruptions in service. Additionally, there is a whole approval process to wade through, and the installations are generally after hours.
God forbid, a bug exists that requires redeployment!
The WSP cycle is a political football in large organizations, and the programmer or developer ends up getting booted.
The above reasoning is why my choice for building pages is to use the Script Editor Web Part with references to code modules in document libraries.
For the next illustration, I will use the code from my “SharePoint 2013, D3 and Dimple Connection” book. All code for the book resides at the following link on github.com: SharePoint D3 Dimple Book or https://github.com/sonisick/SharePointD3DimpleBook.
Feel free to download whether or not you purchase a book. I’ve borrowed a lot of code in 40 years of programming. That’s what Github is all about.
The structure of the extracted zip file looks as follows:
The first level contains visualizations that can run from the two HTML files displayed. The basic D3/Dimple visualization displays from the extracted HTML by just opening in a browser.
The files ending in “WebPart.txt” on the first level are for inserting into SharePoint Pages with the Script Editor Web Part. These files derive from the HTML files in the same directory.
All files on the second level are for placement in a SharePoint document library. The custom code files on the second level: d3RampUpBurnDownChart.js and d3ProdCycleCompare.js are different from those with the same name on the first level files in that they contain extraction code and SharePoint REST and AJAX calls for processing data from a SharePoint list.
In a SharePoint Site Page or ASPX page, the HTML portion requires loading into Script Part Editors. Below is an example of one of the d3RampUpBurnDownWebpart.txt with the CSS excluded for clarity:
Besides the CSS, the web part contains a series of reference to libraries plus a small amount of HTML structure code. The first two references are to internal SharePoint Libraries. The next three are to standard libraries for jQuery, d3, and dimple. The last reference is to the actual code module which does the extraction and visualization. The last four references reside in a document library create called d3. Nothing is special about the SharePoint Document library.
The advantage of creating pages with this architecture is that the deployment consists of a series of routine tasks that any site owner or member with contribute permissions can perform. There’s nothing special in the installation: just create and populate the library; then create a Site Page and a Script Editor Web Part and save. No political or approval headaches are required.
Conclusion
Nothing is easy with SharePoint. The miracle of miracles: Most things work! D3 and Dimple won’t solve all you SharePoint woes, but they will add another dimension to your visualizations, but like everything else, they take work and study.
Happy D3 and Dimpling,
Stephan