The data

For this project I’m going to revisit the house price data we used in our house price meditations. These house price data allow us to explore data that vary over both space and time, and that have interesting hierarchies we will explore.

While data wrangling is an important subject (see for example, this post on wrangling house price data), I don’t want to distract from the dashboard. For this post, we’ll begin with our data compiled.

The data structure is fairly simple. We have columns corresponding to date, metro name, primary state for the metro area (the state of the metro’s principal city), Census region of the primary state (based on Census definitions) the house price index, and the latitude and longitude of the principal city for the metro area. We’ve also computed the 12-month percent change in the house price index, named hpa12.

We’re using the Freddie Mac House Price Index.

I’ve arranged these data and saved them as a simple csv files.

• Metro hpi files hpimetro.csv
• Stae house price file hpistate.csv
• National house price file hpiusa.csv

Here’s how these data look (examining the metros):

For tractability I restricted the number of metro areas, roughly corresponding to the top 20 metro areas based on population.

Here’s a map view of the metros in our data:

Getting started - Data

First we need to load our data. As we’re going to use crosstalk to enable our widgets to talk to each other, we’ll also need to do set up some Shared Data. The shared data can act like data frame in compatible HTML widgets but respond to selections and filters.

In the code below we load the data with metro house prices and create the Shared Data:

df<-fread("data/hpimetro.csv")
df$date<-as.Date(df$date, format="%m/%d/%Y")
# Set up metro data for cross talk:
df.metro<-group_by(df[year(date)>1999,],geo)
sd.metro <- SharedData$new(df.metro, ~geo)

Layout

Our dashboard is going to have several pages. To explore the different layout options we’ll create four pages:

1. A general information/about page
2. A storyboard page
3. A page with an interactive widget we can filter
4. A page with an animated chart

About

The about page is quite important as it is where our new visitors will land. We want to include a brief description along with some hints at what else is in the dashboard. But we want to do it without overwhelming visitors. For this page we’ll include:

1. Short introductory text
2. A map (same as above) showing the metros in our data.

About {data-navmenu="Explore"}
===================================== 

Column {data-width=200}
-------------------------------------

### About this flexdashboard

This dashboard allows you to explore trends in house prices across 30 large metro areas. The metro areas covered are depicted in the nearby map.  The map is colored  according to Census regions.  We picked the 30 largest metro areas based on population. Explore the different data visualizations above.

Column {data-width=800}
-------------------------------------

### Areas covered

# Run this to create map
#```{r jan222017-ex1-map,echo=F}
g.map<-
  ggplot(df[date=="2016-09-01" ], aes(x = long, y = lat,label=geo)) +
  geom_map(data=df.state[date=="2016-09-01",],aes(fill = region,map_id=tolower(statename)), map = states_map,alpha=0.25)+
  borders("state",  colour = "grey70",alpha=0.4)+
  theme_void()+
  scale_fill_viridis(name="Census Region",discrete=T,option="C")+
  theme(legend.position="top",
        plot.title=element_text(face="bold",size=18))+
  geom_point(alpha=0.85,color="black",size=2)+
  geom_text(hjust=0,size=1.75,nudge_y=-0.7)+
  labs(title="Metro areas in our data",
       subtitle="30 large metro areas",
       caption="@lenkiefer Metro population based on U.S. Census: http://www.census.gov/programs-surveys/popest.html")+
  theme(plot.caption=element_text(hjust=0,size=7))
# ```

Some things to note about this page. We want the navigation to be collapsed. The default is for each page to get its own link on the top navigation, but by selecting About {data-navmenu="Explore"} we force this page to fall under the “Explore” link at the top. We also want the map to take up most of the space, so we set {data-width=200} for the first column and {data-width=800} for the second. This ensures the map gets 80% of the available width.

Storyboard

Now we can move on to the second page, which uses a storyboard. Consider the code below:

Storyboard {.storyboard data-navmenu="Explore"}
=========================================

### Map of areas we plot

#```{r}
g.map
#```

### Small multiple, House Price Index

#```{r sm-1-jan22-2017,fig.width=10}
g1<-ggplot(data=df,aes(x=date,y=hpi))+geom_line()+facet_wrap(~geo)+
  theme_minimal()+labs(x="",y="",title="House price index by metro",
                       caption="@lenkiefer Source: Freddie Mac House Price Index")+
  theme(plot.caption=element_text(hjust=0,size=7),        plot.title=element_text(size=10),
        strip.text=element_text(size=4),
        axis.text.x=element_text(size=4)  ,
              axis.text.y=element_text(size=5)   )
g1

### Small multiple, Annual house price appreciation

#```{r sm-2-jan22-2017,fig.width=10}
g2<-ggplot(data=df,aes(x=date,y=hpa12))+geom_line()+facet_wrap(~geo)+
    theme_minimal()+labs(x="",y="",title="Annual percent change in house price index by metro",
                       caption="@lenkiefer Source: Freddie Mac House Price Index")+
  scale_y_continuous(label=percent)+
  theme(plot.caption=element_text(hjust=0,size=7),        plot.title=element_text(size=10),
        strip.text=element_text(size=4),
        axis.text.x=element_text(size=4)  ,
              axis.text.y=element_text(size=5)   )
g2
#```

We start the storyboard page by declaring that this page has a storyboard structure with Storyboard {.storyboard data-navmenu="Explore"}. Note we also force this page to belong under the “Explore” navigation. By adding .storyboard this tells the flexdashboard to arrange subsections on different storyboard panes.

In the code above I included the first three panes (corresponding to the map g.map and graphs g1 & g2). In the full dashboard I actually include 7 panes. The text we include under the headers (denoted with ###) will be included in the story pane navigation filmstrip.

Interactive chart

So far, the elements we have included are standard, and well described in the flexdashboard documentation. These next two pages are more complex. The first, an interactive chart uses crosstalk and plotly to create a dynamic interactive chart in a static webpage.

Crosstalk allows htmlwidgets to talk to one another on a static webpage. What we are going to do is create three plotly graphs and have them linked via crosstalk and include a filter box.

First we need to create the widgets, which are individual plotly charts:

g.map<-
  ggplot(sd.metro, aes(x = long, y = lat)) +
  borders("state",  colour = "grey70",fill="lightgray",alpha=0.5)+
  theme_void()+
  theme(legend.position="none",
        plot.title=element_text(face="bold",size=18))+
  geom_point(alpha=0.82,color="black",size=3)+
  labs(title="Selected Metro(s)",
       subtitle=head(df,1)$geo,
       caption="@lenkiefer Source: Freddie Mac House Price Index through September 2016")+
  theme(plot.caption=element_text(hjust=0))

p0<-
   plot_ly(data=sd.metro,x = ~date, y = ~hpi, height=750) %>% 
    add_lines(name="Index",colors="gray",alpha=0.7) %>% 
    add_lines(name="All metros",data=df,x=~date,y=~hpi,
              colors="black",color=~geo,alpha=0.1,showlegend=F,hoverinfo="none") %>%
     layout(title = "House Price Trends by Metro",xaxis = list(title="Date"), yaxis = list(title="House Price Index"))

p1<-
   plot_ly(data=sd.metro,x = ~date, y = ~hpa12, height=750) %>% 
    add_lines(name="Annual % change",colors="gray",alpha=0.7) %>% 
    add_lines(name="All metros",data=df,x=~date,y=~hpa12,
              colors="black",color=~geo,alpha=0.1,showlegend=F,hoverinfo="none") %>%
     layout(title = "House Price Trends by Metro",xaxis = list(title="Date"), yaxis = list(title="Annual % Change in House Price Index"))

g.map is a ggplot2 graph while p0 and p1 are plotly graphs. We will apply ggplotly to convert our ggplot map into a plotly thing.

Once we have the graphs, we can combine them using the crosstalk function bscols and include a filter_select to filter the charts. The code is not very long:

bscols(widths=c(2,6,4),
  list(filter_select("metro", "Select metro to highlight for plot", sd.metro, ~geo,multiple=FALSE)),
  subplot(p0,p1,nrows=2,titleY=T),
  ggplotly(g.map)
  )

The bscols function first allocates our graphs over the page with widths. Next, we include a filter_select that uses the SharedData sd.metro (discussed above). We set multiple equal to FALSE so that only one metro can be selected at a time.

g.map2<-
  ggplot(sd.metro, aes(x = long, y = lat,frame=geo,label=paste("\n\n  ",geo),color=geo)) +
  borders("state",  colour = "grey70",fill="lightgray",alpha=0.5)+
  theme_void()+
  theme(legend.position="none",
        plot.title=element_text(face="bold",size=18))+
  geom_point(alpha=0.5,size=1)+
  geom_text(hjust=0)+
  labs(title="House price trends around the U.S.")+
  theme(plot.caption=element_text(hjust=0))

p2<-
  ggplot(data=sd.metro,aes(x=date,y=hpa12))+
  #geom_point()+geom_segment(aes(xend=date,yend=0))+
  geom_line(aes(frame=geo,ids=date,label=geo,color=geo))+
  scale_y_continuous(labels=scales::percent)+
  geom_line(data=df.us,color="gray",linetype=2)+
  #scale_fill_viridis()+  scale_color_viridis()+
  theme_minimal()+labs(x="",y="Annual House Price Appreciation y/y %",title="Annual Price Growth")
#+    geom_text(data=d3.m[date==median(d3.m$date)],fontface="bold",y=16,size=8)

p3<-
  ggplot(data=sd.metro,aes(x=date,y=hpi))+
  geom_line(aes(frame=geo,ids=date,label=geo,color=geo))+
  theme_minimal()+labs(x="",y="House Price Index",title="House Price Index")+
  geom_line(data=df.us,color="gray",linetype=2)


subplot(subplot(ggplotly(p3),ggplotly(p2),nrows=2,titleY=T), ggplotly(g.map2),  nrows = 1, widths = c(0.35, 0.65), titleX = TRUE,titleY=T) %>%
  hide_legend() %>%
  animation_opts(2000,transition=500) %>% layout(title="House price tour, solid line metro, dotted line U.S.")