My Drupalcon Amsterdam notes

Here are my notes from Drupalcon Amsterdam, culled from Google Docs and minimally formatted. They're likely to be of more value to future generations than right at this minute.

Edit: OxDUG notes

Tuesday 30 September

State of Drupal: Dries Buytaert

China/Japan: held back by translations/leadership
[Going outside any comfort zone (e.g. Western Englishspeaking) means learning about deficits one never knows one has had all along]
Getting community to write Driesnote a good way to unearth what we don't know

Let's talk about public goods: none-excludable, non-rivalrous
Public and OS in general is one
Other public goods: the Ridgeway - road since 3000BC, created by volunteers
Volunteers->business->government [really?]
Volunteers are (still) crucial

Tragedy of the Commons. Still a commons in Boston!
Caretakers of the commons ("I look after the land I use") vs Free-riders of the commons ("George will do it")
Helping a little bit is a no-brainer; why don't we?
As you grow, the cost of contributing increases & the benefit decreases
e.g. we used to do a slide of core contrib photos; now it's just a tag cloud of names

Freeriders aren't bad: they evangelise; they are potential caretakers.
Yet the caretaker/freerider ratio decreases with size; how do we make it increase?
Common tactics: Privatization, legislation, altruism, taxation, social capital, reducing costs, privileged groups

What tactics work for us? Well, what are the caretaker costs?
1. d.o has been spun out of the costs, delegated to DA
2. Removing NIH has spun libraries out of costs, delegated to Symfony
3. But core got bigger as a result
Need to start thinking about simplifying until we can simplify no further.

Can we get companies to see benefit in hiring core contributors?
A "contributor" is not always an individual: also web agencies and end users
Can we bring companies into commit messages? [!]
Marketizing [very limited types of] contribution
Adverts on the d.o homepage?
Can we set up a complex rankification device, to balance contributions against each other?
[Is this not just "We need to do something!" "Here is something!" "We need to do it!"]

What about Drupal end users
Hiring new talent is hard: 92% pool too small; 82% will hire in next 6mth
Drupal jobs: flag big contributing companies to show they're good to work for
Individual user profile improvements [Open Badges!]

The cycle of incentivization can help address the problems people came up with to talk about: innovation, release schedule, funding core dev, sustainability, developer burnout

D8: we had zero blockers
Found one more on Mon!
Talked about it Mon p.m.
FIXED IT early Tue.

Beta 1 releases THIS WEEK!
Still very beta
Core contribs stand up; but also - given discussion so far - their companies also stand up.

D8 crash course: Larry Garfield

Controllers can still return strings or render arrays: but they can now also raise exceptions.
Controllers get a request object - the Symfony way (by introspection) - (... Request $request) { ... and URL templates in routing.yml

Drupal is about: building tools; then wiring it into the system
So: Extend a base class/interface; tell Drupal about it

Module is a folder with a YAML file - and that's it.
Make a page? Extend a base class, wire it up.
1. Make a controller extending ControllerBase - PSR-4 means it's automatically available everywhere.
2. routing.yml registers your URL

Templating: ./template/{theme_key}.html.twig
* config/install/{module}.settings.yml with default variable values on install
* config/schema/{module}.settings.yml for schema-like data; can e.g. turn on translation
Form to manage config: src/Form/{FormClass}.php
* Extend ConfigFormBase
* defaults: _form: Drupal\{module}\Form\{ClassName} in routing.yml
Menu items: add default values for URLs; validate URL values.

Plugins; plugin types. How blocks work.
D7 extensibility - info hooks, but little else in common. Ctools, features etc. have all their own system.
D8 extensibility - User-configurable logic? Plugins. Interface, manager class, done.

Can place blog multiple times now
Overriding methods extends the block configuration form

State tracker: write a class
Register in services.yml
Controllers aren’t registered on a container; but controller classes have a static method
On the block, we pass the same service through, implementing the ContainerFactoryPluginInterface

Entities; content entities
Controller method argument autodetected as per Symfony: (NodeInterface $node)
Nodes are actual objects with methods! $node->isPublished()
Even $node->body->value [entity_metadata_wrapper no longer needed]

When to use annotations?
What something is? Annotation
Where something is? YAML
(A rough approximation)

Smaller classes/methods
Stupid glue-code controllers: no 300-line page callbacks!
Logic in services instead
Not a DB query in sight?
* Store information? Entities, configuration, state.
* [Reading in bulk? EFQs?]
* This way you can move [select bits] to NoSQL if you need to.
* If it's hard to test, you're probably doing it wrong

Open Badges: Rachel Lawson

You learn things every day that can be useful to your work
But the evidence of what you learned is scattered, or doesn't exist

Scout badges; for Drupal? Badges on laptops. But how do you know the badge is "valid"?

Mozilla Open Badges
Standard for showing evidence of achievement
Issuer, Displayer, Backpack
Badge power can collect badges in different collections: only some public

Badges are just JSON, with salt and encrypted values

CPD - wanted to issue badges
First thing: explain to people.

One website can interrogate the backpack provider and get badges e.g. from other providers.

To build in d.o:

* Achievements module - already present on d.o
* amobb module
* Implemented certain one-off badges and also badges when they get e.g. above a certain number of replies to forum posts

But you don't have to use this method - badges are just images and JSON.

Profiles are being improved
But the DA can't do everything
And this federates away the hard work to many loosely coupled providers

Many organizations should have the chance to do certification
This way we don't e.g. hardcode only one big player being able to show certification
Not just Acquia v Lullabot - also Drupalcamps.

Contrib module update: various (presented by Michael Schmid)

Rules: Josef Dabernig

d8rules: looking at crowdsourcing; you can contribute and get a rule(r) - one for each E45 (cost of a dev in Wien)

Panels: Jakob Perry

Page manager project
Layout module - decision needs to be made: right now, it's in contrib
(frega's) layout plugin
Maybe ctools not required in D8?
David Snopek made a Panels Layout Variant - PoC broken by D8 core pivots!
Looking at a migration path...
Similar to D7 in appearance.
Spark initiative - still going but not coupled to Panels development

Media: Dave Reid & Yannis Karampelas

Been problematic e.g. WYSIWYG doesn't always work
D8: Fixing issues from ground up
Learn from Skald
Split things up!

Entity Browser - separate project. General entity browsing tool
Media entity - "not everything is a file"; works heavily with fields, straightforward remote resource (e.g. oEmbed)
File entity - does so much right now, that needs to be splitted out
Fallback formatter - For any field with more than one formatter - if one file formatter returns nothing, define fallbacks
Weekly scrum meetings, g.d.o/media - contrib sprint on Friday

Embedding of images now works in core
Estimate: when it's done; they don't have funding (yet)

Search API: Thomas Seidl

Architecture should remain unchanged
Looking at a way to solve problems more efficiently based on D7 experience
Focus on flexibility and UX

Flexibility: indexes won't be restricted to a particular entity type
UX: revising interface, standardizing terminology
Planned: simplified wizard; docs via Tour module
Status: existing func ported; a lot of improvements still to do; but some people are using it in production
D8 & Solr? - only Search API.
Other backends: DB Backend, Elastic Search (partially)
Facets: we don't know yet.
Multilingual search: all in Search API framework.

Commerce: Bojan Zivanovic

1.x: 50,000 live installs
Getting off the island: so develop libraries and separate out

commerceguys/intl - currency formatting e.g. per-locale Euros formatting
D8 uses Unicode CLDR for country list
Commerce is going to use it for dynamic currency formatting based on language

commerceguys/addressing - replaces AddressField
AF had to have manual address fields; but there's a Google dataset

commerceguys/zone e.g. "European Union" or "Europe and Middle East Asia"
For taxes or shipping: reduces those rules

commerceguys/tax: most ambitious. Bring in Commerce VAT and Commerce UAT
Cover as many common use cases as possible

Module on github; using Travis Build Bot to run own tests & d.o bot doesn't run Composer

BoF: freelancers unite (briefly!)

(No notes)

Wednesday 1 October

Keynote: Cory Doctorow

Motif about freedom in software: about whether information wants to be free
Information wants not one thing; if it did, who cares?
People want to be free, and cannot in the inf age without free software.

Computers are everywhere: your car is a computer you sit inside.
Planes, walkmans, iPods, hearing aids.
"Can I have that powerpoint?" "I'm charging my laptop." "I need to charge my leg before the flight." "... Yeah, it's all yours."

Alchemy looked like science; but alchemists never got the reality check of e.g. peer review. So it did not progress: everyone had to learn from scratch.
Security only works if it's open; otherwise all you've got is a system that works on people stupider than you.

In a sane world, we'd only accept the FLOSS model for *everything* e.g. building conference centres:
"We're not going to tell you how we calculated your load stresses because that's what gives us a competitive edge."

Every problem has a computer at the middle of it; but lawmakers do not understand what that means.
They think a computer is a bit like any other product: inherently regulatable.
You can't say "make me a computer that's incapable of running a program that's harmful".

But we still try: Region-specific DVDs; Apple-signed code. It's become a feature of law.
If you're still making the same mistake about the internet after all these years, you're stupid.
Laws to prevent revelation of bugs in DRM systems: means that's the most insecure aspect of your computer.

Leaking of e.g. recent photos comes from people waging war on the freedom of computers this way.
If you don't (aren't allowed) to know what your browser is doing, you're at ultimate risk.
If you don't know what your remote-access home thermostat is doing, the power company can collude with the government to turn the heating off if your phone is detected at a protest.

DRM is disempowering encryption; but in doing so, it has to give everyone the same keys and hope that not one in a billion of us works out how to use them.
DRM not like ordinary encryption: fundamentally disempowering and controlling. This permeates the craziness of its architecture.

We need to make this a big social and political problem. We need to take this to parliaments and congresses.
I have hope, if not optimism.
Not enough to have free software; we have to have activism.

Three strikes rule: what does that *mean*?
Martha Lane Fox asked PwC about digital inclusion
Social housing estate experiment: given them free internet access.
EVERYTHING CHANGED - all measures we use to measure QoL
Better education; more likely to go on to tertiary; more socially mobile; more politically aware; more likely to vote.
That single wire gave everything: freedom of speech, a free press, improved health.
The Secretary of State could at his discretion "if you watch TV the wrong way, we cut all that off from you."

If we can't trust these tools, we've lost this fight.

(While a long rambling statement followed by "what do you think" is technically a question, it's not a good one.)

I love cool hardware, but if you're buying it from the fruit company that's normalized DRM then that's part of the problem.

Content staging: Dick Olsson

Learn from UUID etc. in D7
Revisions, conflict, dependency, ad-hoc/continuous, bidirectional, REST

How to build:
Core: serialization srvices, entity
Contrib: Multiversion & Relaxed Content

Multiversion: tracks update sequencies to make dependency management better (Which module is adding to this node?)
Look at it more like a NoSQL database document object and replay changes; tracking revision trees in a git-like way
UUIDs based on MD5 hash of changes. Easier to detect conflicts without deep inspection or asking network
Uses JSON as its serialization format [? not valid JSON on slide, though]
CRAP not CRUD (Archive and Purge: we never delete.) This means we can "replicate" deletions.
"Compaction" available as a real deletion of old revisions.

relaxed.module - restful/"relaxed" JSON API Endpoints for all content entities and file attachments, comparing revisions, start/stop replications etc.
Drush plugin

deploy.module - simple UI focussed on staging

How replication works:
1. compare UUIDs
2. get target site checkpoint ("when was last replicated")
3. get changes since checkpoint from source
4. pass all changes to target site
5. collect all missing revisions from source
6. POST to bulk ops on target
7. reset checkpoint
A lot less comms over HTTP than in D7, because HTTP is inherently unreliable

Does not have a live demo yet? But good test coverage.
Lots of functionality "already there", so workable

Reusing protocols: revision/conflict detection model same as git/CouchDB's. Don't use storage; just protocol.
CouchDB figured out HTTTP replication; so we should just use that!

This leads to us being "CouchDB/PouchDB/TouchDB"-compatible
Can support Headless Drupal: AngularJS PouchDB D8

Drupal 8 CMI: David Strauss & Matt Cheney

Configuration vs content
Two different pillars of information
Historically Drupal doesn't care: all in the DB

Features is a 2009 solution for this. Some might not like it, but it works.
But features is contrib - not fully integrated - and not intended to be full solution for storing D7 config
UUIDs, dealing with state/changing data structures etc.

Proudly Invented Elsewhere: not NIH.
So we trashed Drupalisms: No New Parser. Don't use info files.
Started with XML but moved to YAML parser after move to Symfony.
Symfony's Config library was good - but no good GUI support.

Inspiration: Jenkins does diskconfigGUI
Goal: minimize impact on basic users; don't get in their way if they don't really use CMI

Multiple invironments, argh: want to keep config & e.g. code integrating with a view in sync
But D8 CMI only has one other extra step: like Features Revert, can be done with Drush

Wanted GUI to be extremely basic. Really clear on how to use.

Features - complicated semi-custom files and the diffs aren't guaranteed clean.
CMI goal: should look beautiful on disk. Deterministic in e.g the order of fields
CMI: and - that's it.

Live demo: Pantheon offfering D8 beta
Changing site name on dev; commit; then move to test and run "Synchronize"

Drush7 - six commands for (local-with-vim) edit, export, get, import, list, set
$site_name = Drupal::config('')->get('name');
->set() and then ->save() - Symfony model - make multiple changes at once and transact.

Only for this one environment? Use ::state()->set('key.with.dots', $var)
Still got global $conf in settings.php [!!]

Some concepts available for D7. But suffers from limitations like Features.

Multiple languages e.g.{file.key}.yml overrides

Features? In D8, a feature will be a bundle of CMI files. What it was originally supposed to do - migrate packaged functionality.
Other projects: config_readonly (locks config); config_inspector, _log, _tools

Future-proof your Drupal 7 site: Dave Reid

Make informed choices now; ease pain when switching to D8. Backport all the things!

Spark distribution - make lots of new modules in D7 with intent to bring into D8.

navbar.module - replace admin_menu and/or toolbar. Responsive-friendly. admin_select: different toolbar-y module per role
quickedit.module - inline editing
ckeditor.module - official WYSIWYG now
responsive_bartik.theme - A nicer responsive Bartik
HTML5: elements.module, html5_tools.module
Views: well, yeah, everyone uses it - but it's in D8 core!
admin_views, VBO (=lite in D8 core)
Views Responsive Grid
Responsive Tables
Breakpoints module, supports:
picture.module - responsive pictures
joyride.module - Tours around the site - demos etc. "How do I X"
instantfilter.module - Module page filtering. Like a tiny modulefilter.module, but then it supports lots of other lists of stuff
Simplified menu admin
Escape admin - what overlay was intending to support
Caption_filter - uses HTML5 attributes, converts to HTML5 elements on the frontend - and is very nice

New field types in D8 core that you can use in D7:
entityreference - NOT nodereference or userreference. ER does e.g. configuring cascaded deletion of related nodes
telephone.module - NOT phone. This is more similar to D7. Does very little validation: PHONE NUMBERS ARE HARD.
email.module - D8 is a lite version of this
date.module - BUT - use Date (ISO format) for forward compatibility

uuid.module - whenever you need to support UUIDs. You don't NEED it right now.
render_cache and entity_modified - cache rendered node even for logged-in users; entity_modified expires it on mod
composer_vendor and composer_manager

Plugins (Joachim documented it)
Emulation in D7?
* Hooks
* CTools plugins - can be quite confusing
* Roll your own - use interfaces and classes

Other stuff
entity_view_mode - started in D7 but really supported in D8.
RESTful web services: restws and gizra/restful
Translation: entity_translation and title.module

More things: PHPUnit, Symfony, Backbone.js, Underscore.js, Guzzle, JS local storage
Distribution - for demo only, really - pred8.profile

Things you should probably stop using
First check

Symfony2 best practice: Stefan Koopmanschap

No notes: standing room only!

BoF: Open Drupal

(No notes: see

Thursday 2 October

Lightning talks: hosted by Michael Schmid

Drupal & Bitcoin: Jonathan Brown

Shared autonomous ledger; consensus among peers; orders transactions to avoid double spending
Git similar to blockchain, but usually pushes to a blessed server; but Gitchain
Twister: Twitter implemented on a blockchain
Bitmessage: whatsapp on a blockchain
Blockchains can circumvent censorship (?)
Drupal bitcoin projects: cointools; also startups

The Unforeseen: Steve Parks

Large orgs terrified of risk, the unforeseen. Tends to come from some mythical terrible project in their history
So they put in ALL THE RULES. That'll fix it.
Worse than that are the people who completely stick to the rules - all grinds to a halt.
Welsh translation of sign
Rules become crazy: Zippez prudement!
Gaping void of trust
Need proactive planning and TRUST
We have to accept risk and go in with the right training, tools and attitude

Druphpet: Aleh Kot

Many dev stacks available
But for big sites not enough: need debuggers, profilers, git standards, code quality, automation etc.
Huge amount of tools production projects
Druphpet fork of Puphet: w/Samba, Solr, other features
Use Vagrant
Puphpet internal storage is JSON
Customize this JSON
Install Vagrant plugins and up

Continuous Delivery: Michael Godeck

As with Agile, CD's main aim is to mitigate risk
Continuous Integration a cornerstone of CD, but build/deployment pipelines are central
Break up build into stages
Build pipelines are about rehearsing delivery

D8in8: Stella Power

Internal competition to get entire team learning Drupal 8
8 Weeks to complete
8 Minutes to present
Judged by peeers
8 points to distribute
Managed to get responsiveness, photo gallery site, trivia game based on custom entities and node.js
Winner: Andrew Macpherson: got core commits incl a Field API Beta blocker; ported modules etc.
RUN A D8in8 competition!

Drupal 8 console: Jesus Manuel Olivas

Use Symfony Console component
CLI-interactive creation of e.g. modules, like with Symfony bundles or entities
Generate a controller; generate a form.
Generate routing
Crell is to blame for this ace tool!

Drupal security: Lee Kelsey

Why does security take so long, especially to percolate through distributions?
Security team hoping to be more transparent about their process and how it affects you.

Better UX through prototyping: Roy Scholten

Important to create big IA of sitemap, but static deliverables are static. Wireframes can't tell the whole story

Prototype: why?
Interactive deliverable easier to understand; results in better feedback.
Behaviours: can be responsive from the start; manage expectations better. Shared quirks!

Quick iterations
Easily shareable. People can use it on their own device.
Forces you to make links really work.
Reduces risk
"A prototype is worth a thousand meetings"

Prototype: when?
Process diagrams - so many of them out there.
Follow a UX-driven one: do all your learning and brainstorming on the user experience first, and let that lead the prototype.
"Double diamond" - prototyping in second and third quarters, separately on definition and execution phases (not sure about first).
More honest representation of the process - "The Squiggle". But that's why you want to iterate.

Prototype: what?
Focus on essentials e.g. login
Focus on what it does, not how it looks

Prototype: how?
Paper prototypes, interactive.
What kind of sketch -> What kind of prototype
Paper - but cut up bits.
Then wireframing tools but make clickable; Keynote?

Live HTML? It's not Drupal
But can still use your own base theme - have it in your wireframe environment.
Define executable PHP scripts that bring in the CSS.
Focus on building screens - DON'T USE PATTERN LIBRARIES/REUSABLE COMPONENTS - yet. Too abstract, overhead.

Git version control and automatic builds: auto-upload on master

Prototype: who?
Teamwork, not just info architect or designers
"UX designer + UX developer"

Teamwork means you need availability, can encounter bottlenecks.
NOT the branding. It's a functional prototype. Be clear. "It's not the branding design. It's ALLOWED to look a bit grey."
Can become unwieldy
Try it out, start small

- Questions -

You CAN respond to an existing third-party agency design with a prototype. Appeal to their wallet. Review the design and suggest prototyping. Be positive. If they're too wedded to the design, that's maybe a cue to not take on the project.

Tour de Drupal BOF

(No notes)

Ember.js and Headless Drupal: Mikkel Høgh

Ember.js - single page applications. Have browser not reload all the time. Inevitable model, driven by expectations.
Different philosophy from e.g. Backbone. "Ambitious". Might not be for your blog.
Full-stack MVC in the browser. MVC a bit old school - model meant to persist across user interactions in a desktop environment.
Convention over configuration: less boilerplate code. e.g. naming conventions.
Fully class-based object model (close to prototype-based e.g. Ember.Object.extend({}))

[Has a *server* component? Is that just holding static HTML?]

Two-way binding: on all Ember.Object inhstances. But in practice any object binding. TemplateModel.
Computed properties: properties as functions. Does mean you have to use .get() and .set() everywhere.

How constructed.
Dom/View/Template/Model/Controller/Route - all communicate via events, rather than directly coupled layers

Entry point is routing.
Naming convention: route name-> [CamelCase]Route and [CamelCase]Controller. So just define a route and a template. Every layer in between is optional with fallbacks.
Router reacts to route change
Loads data and injects into controller
Knows about hierarchy of routes

Full model system
Knows state of all data, which attributes to be saved, what has been persisted to the server.

Holds and manages data; handles useer actions and provides context for template
So detect when user presses cancel object
Any data changes can be rolled back as long as the same context is preserved.
Controller can direct - transition - to a new route

Handles UI state; has direct DOM access; *can* handle user actions if it's not about data but about changing visuals

Handlebars - very Twig. Handles two-way bindings. Not really much logic.

Why with Drupal?
Drupal handles ORM really well, gives a good UI
Drupal good at REST services; services, restws and then into D8
Often need a CMS anyway: info pages, documentation, support pages etc.
Drupal can give you Commerce
Good to have a backend you know already

How to integrate
Ideally REST/JSON all the way
Ideally D8
But not always possible
If you need to override at the Extend DS.somethingorother he changed the slide as I was typing

SEO? - it's a long story.
cors.module - Cross Origin Request Service. Works around the cross-domain policy.

[Live demo]

Ember CLI

Decoupled Drupal with Silex: Larry Garfield & Hagen Last

CMS for OTT solution
Needed support structured metadata, UI-managed
Publishing workflow, curated content: banners, homepages, lists of "stuff"
Data sync woith Ooyala
APIs working at scale
1000s of movies, 10000s of users
Support website and apps

You know what's good for this? Drupal 8! But, oh, early 2013.
D7 with Services. But Services isn't actually REST; it's more RPC. Architecturally it's fighting against D7.
RestWs? In early 2013 it was still fairly new.
Pure Symfony? We lose a lot of Drupal's out-of-box capabilities.
Silex? 3 times faster than D7 on "Hello, World", prior to ANY optimization. But it's a microframework, so even less out of box.

D7 + Silex?
Embed Silex *in* Drupal?
Leverage the right tool for the job.

Drupal data is Drupals
Editorial structure != API structure
So put Elastic Search in the middle?

XML =import=> Drupal =editorial=> Elastic Search =read-only=> Silex =web=> client browser/app

Importing - Migrate is meant to run from a user trigger. This was a PUSH import.
Feeds - wants to map things to one object per thing. We wanted to map to many. Got a lot of moving parts we didn't want the overhead of.
Services - kind of clunky & not really RESTful.
Parsed XML with a library called Querypath (module for it too)
Closure-based mapping objects - potentially go to mutliple nodes
HTTP - well, it's a cheat: POST-only, no REST
3 page callbacks, common code, simplehttpauth to restrict access over SSL.

XML data wasn't very good, so had to talk to a third-party service. Rotten Tomatoes!
Use Guzzle - this replaces drupal_http_request() in D8, which is DAMN hard to debug.
Guzzle lib for Rotten Tomatoes - released on Github!
Rotten Tomatoes module also released!
Map nodes by IMDB ID

Editorial: main interface is Views with filters
Editing etc. tabs built with Panels
Custom form alters to e.g. log message only required if as a reason for rejection
Overriding form edit pages with panels is painful - needed form alters e.g. around the required fields.

Export to Elastic Search
Lots of preprocessing needed: Drupal structure is NOT the same as the API structure. Reformat once, on the way in.
Also contrib on ES quite weak.
Rules has too many moving parts.
Exporting some nodes also requires others: so programmatically publish assets.
Another mini-custom OOP system, decoupled from Drupal but within it.
Complex rules about what can be published: so don't do on publication - takes too long. Use queuing and cron.

Silex with Hypertext Application Language extension to autodiscover from one JSON resource to another.
Extends JSON to provide hypermedia links. D8 uses this out of the box
Returning objects within Silex, way back to the kernel
This is the same architecture as in D8
HAL Browser - single-page app to browse any arbitrary API

High availability:
Silex is stateless - spin up several and load balance
Elasticsearch clusters really, really easily
Varnish caching = FAST

Closing session