Implementing Marketo progressive profiling in Javascript

If you’ve used Marketo’s progressive profiling engine, you know it has a bunch of limitations. It:

  • Doesn’t work between different types of forms. So a user who tells you their job function on a webinar or event registration might not have that auto-filled on a Talk to Sales form.
  • Is inflexible; all you can do is specify a list of fields you’d like filled in, and how many blank spaces to present at once. That can be frustrating if you’re staging your leads such that you need different amounts of data at each stage, e.g. First, Last and email, then Company, Job Function, Job Title, and Revenue, then Interest.
  • Only works on Marketo’s hosted landing pages. Some users want more control over their landing pages, are using a CMS that allows for easy form management, or need to ensure proper integration with analytics services, which Marketo doesn’t support.

Fortunately, progressive profiling can be implemented much more flexibly with a bit of Javascript and the Forms 2.0 API.

The code below:

  • Saves the user’s information in a cookie, locally, and uses that to pre-populate any form on your site with matching field names. (This assumes you are not collecting sensitive information through Marketo, such as a personal mailing address. If so, you probably shouldn’t store this information locally.)
  • Hides any fields that are already filled in (reducing perceived workload), unless they are textareas, since textareas are also used for comments that should not persist from session to session.
  • Uses fieldsets, defined in Marketo’s form builder, to break the form up into sections that are displayed on separate visits. This means that if you want to collect information in three stages, for example, you can break your form fields up into three fieldsets, and the user will only see one at a time. If you want a field to show up all the time, don’t put it in a fieldset.

Some notes: The code relies on the extremely useful jQuery Cookie library by Klaus Hartl. This is only the first version, so there are probably bugs and cases I haven’t considered. You’ll need to assign values to marketo_instance_id and formid. On our site, we use CSS so that the fieldset itself doesn’t add additional styling.

The code

Start by loading the form using the typical MktoForms2 methods. Within the load function, define onSubmit so that it grabs all the form values using the .vals() method as soon as a form is submitted (but before success). Remove the formid and munchkinId properties so they don’t interfere with other forms, and store a stringified version of the returned object in a cookie.

MktoForms2.loadForm("//", marketo_instance_id, formid, function(form){
var vals = form.vals();
delete vals.formid; delete vals.munchkinId;
jQuery.cookie('formdata', JSON.stringify(vals), { expires: 30, path: '/' });

Next, add some work to happen on the whenReady event, which MktoForms2 fires when a form has been loaded from Marketo and fully rendered. First, check to see if the formdata cookie exists. If it doesn’t, don’t do anything.

Then grab the formdata cookie, parse the contents into an object, delete the formid and munchkinId again as we did above, just in case, and populate the form using the Marketo vals() function.

MktoForms2.whenReady(function(f) {
if (typeof jQuery.cookie('formdata') == 'undefined') {}
else {
var formdata = jQuery.cookie('formdata');
formdata = JSON.parse(formdata);
delete formdata.formid; delete formdata.munchkinId;

Once the form has been populated with cookie data, hide all the fields that are already complete.

jQuery('fieldset input, fieldset select').each(function(){
if(jQuery(this).val() != '')

Finally, look through all the fieldsets. Is any of them completely blank? (We test for this by concatenating the values of all the fields within the fieldset). If yes, remove all the following fieldsets.

for (i = 1; i < jQuery('fieldset').length; i++)
if (jQuery('fieldset').eq(i-1).find('input,select').val() == '')
jQuery('fieldset').eq(i).remove(); i = 0;

Note that this code doesn’t “count” fields of type textarea for the calculations of what is blank or what to hide, though it does save the value.

It’s worth considering whether textareas should be treated as other fields are (which is easy to do, just add textarea where you see input together with select, or whether data left in them should not be stored, since it’s usually things like comments which aren’t useful to save. The latter would be harder to do since .vals() doesn’t give you a way to exclude fields based on type.

Emotional laziness and the discomfort of working on important problems

Here’s an essay by Paul Graham, popular today on HN, in which he talks about “good” and “bad” types of procrastination.

In short, “bad” procrastination is when you work on things that are unimportant in order to avoid working on things that are important. “Running errands”, he calls it, in order to avoid, say, writing your novel.

One of the major reasons people do this is that working on important things is scary, especially if they are also difficult things, which is often the case. So it takes a real effort to focus on them.

But what kind of effort are we talking about? Sure, there’s an intellectual component, but that only happens once you are engaged in the task.

I think that the upfront labor – which we don’t talk about – is emotional. It’s getting control over the impulse to work on unimportant things, things that may even give you immediate satisfaction, in order to focus on what’s truly important.

This happens to me all the time. I put off a very important, but difficult, task, in order to do something that is fairly easy to do and gives me an immediate feeling of satisfaction. I certainly don’t consider myself a lazy person; after all, I get a lot done! But in fact working that way is probably a form of laziness, because I’m not willing to delay the gratification of completing that task.

Another variant, which I think is the most important one within an organization, is unwillingness to say “no”. It’s very satisfying to say “yes” to all requests that come your way. At least, I certainly find satisfaction in what I view as helping others, especially if I can do it really quickly. But becoming interrupt-driven makes it very hard to do some types of good work, because you can’t sit down and focus on a problem. There’s a certain amount of urgent work that has to get done, but by doing it all the time, you create the appearance of helping others, without, perhaps, actually doing that.

A third variant, which I am not totally sure about, is working constantly. Yes, of course it’s important to be engaged and responsive to your customers or your coworkers. But if you never step away from a problem, is it actually possible to come up with good solutions to it? What if you never let others (your reports, for example, or your colleagues) step away from the problem? “Just walking away” seems to be very important for good problem solving. But I certainly find it extremely difficult to do. Maybe you do, too.

A related problem, by the way, is caring too much about what people think. Everybody wants the good feelings of being socially accepted. We’re programmed for that, after all. But often doing something interesting or valuable requires an initial period of not being accepted or understood. It’s important to be OK with that.

The unifying theme of all of these, and what makes it emotional labor, is that you have to be willing to be uncomfortable in order to solve these problems. Just as a good workout might make you sore, or a hard problem can be intellectually frustrating to work on, working on the right things might make you feel things you don’t want to feel. It’s important to develop the ability to be uncomfortable, and maybe to make others uncomfortable, when it’s appropriate.

Important things can be difficult to measure (but we should still try)

I keep track of lots of numbers when I go to the gym. For example, I can do x bicep curls at a time with an x-pound weight. Or I can run a mile in x minutes. Over time, I’ve seen the numbers generally go in the direction I want them to; I can run about 35% faster than I did three years ago, for example.

Measuring exercise progress is nice because I have time to do a broad range of exercises (weightlifting, running, swimming) and I know that my improvements across the board generally correspond with my goals in going to the gym: having better physical health and more energy.

But what if my goal is to become a more educated person? Well, I could track that by maybe tracking the number of books I read per week. Of course, I’d have to make sure I focus on the true classics. What determines this?

Well, culturally I suppose there’s an generally accepted canon, including ancient philosophy, well-known novels, and plays that are cultural touchstones. Maybe I should track how many of those I read. (What about what I get from them, though?)

Or I guess I could read books on how to code, or how to speak French.

Well, is it better to read more, shorter, books, or fewer, in-depth ones?

Is reading books even a good measurement of what it means to be “educated”? Maybe my goal is not defined well enough to be measured properly. But it’s probably a worthy one.

What if my goal is even more abstract, like becoming a better person?

There are cases where you can measure exactly what you are trying to improve. If you are trying to get a higher click-through rate on a page on your site, it’s really easy to measure that. But as the goals get more abstract, they’re much harder to measure. The more that happens, the more a measurement is simply a proxy, a simpler standin, or a model, for what you actually care about. It’s an easy question swapped out for a hard one.

In writing this article originally, the title was, “as goals get more important, they’re much harder to measure”. There are some trivial examples of this, like GDP, which is a very poor measure of economic success, especially for poorer countries, and yet one that people tend to focus on. But, on the other hand, there are some incredibly important goals that are actually easily measured by very simple numbers: things like infant mortality rates, malnutrition, and even deaths due to preventable diseases.

Given that it’s basically impossible to improve without measuring things, and given that there are at least some things we can say about what matters to us that is quantifiable, I think it’s still worth it to try to measure most things we care about.

Why isn’t mesh networking more popular?

Mesh networking is an exciting idea (quick explainer below if you’ve never heard of it), the technology (largely) exists to do it, and there are some pretty clear use cases related to routing around entrenched internet gatekeepers (whether they are governmental or private).

Perhaps most importantly, the idea of mesh networking seems to be a fit with our current trend toward decentralization, where there’s money to be made in providing infrastructure for peer-to-peer networks, rather than for providing services directly. Think of Uber, Airbnb, or Bitcoin.

And yet this Google Trends graph shows declining interest in mesh networking. Why?


As a quick recap, connectivity over a mesh network is like a grid of streets as opposed to a single interstate highway.

If you’re on a mesh network, a request from your phone can go to a cellular tower as it normally would. Or, the request goes to someone else’s phone nearby, and that person’s phone then gets the data from a cellular tower. Or, through some other device or chain of devices. Advantages include:

  • Vastly increased reliability, and potentially performance, of the network by providing many alternative paths for data
  • Resistance to network shutdown or damage (by governments or others) for the same reasons
  • Increased coverage, since areas at the edge of a cellular tower’s range (for example) can be connected with nearby devices as intermediaries

Of course, the technical problems of building a network that can heal, adjust and optimize itself in real-time are enormous, though many of these challenges are already being addressed.

There are a couple of recent-ish developments, too. Apple released the Multipeer Connectivity Framework in iOS 7. Though limited in its application so far, it essentially allows iDevices to create their own mesh networks. (Android doesn’t appear to have an equivalent, though I’m not sure).

The most popular application using that capability right now is FireChat. The company that makes FireChat has also released Open Garden, which is a mesh networking tool for desktop devices. Fred Wilson loved it (in 2012).

And… that’s it?

It seems as if there a few ways that mesh networking could get popular:

  1. Network congestion and failure could become such an issue that consumers are driven en masse to a different way of doing things. For example, cellphone networks become unusable or a giant ISP starts throttling traffic to non-preferred providers, and consumers get frustrated. Killing net neutrality rules seems like a good way to make this happen, but only if consumers are made aware of whose fault the delays actually are. In general, in my personal experience I’m seeing the opposite happen (the networks I use every day are getting more satisfactory, not less), but others have had other experiences.
  2. Alternatively, ISPs could somehow become so desperate for better bandwidth utilization that they are willing to consider a technology that would eventually render them even more commoditized than they currently are.
  3. Mesh networks are deployed in developing / authoritarian countries with significant impact. I don’t see people being aware enough of what’s going in the developing world to care about mesh networks that are successful there. But perhaps some kind of political change becomes possible because of the technology, and we hear about it.
  4. I think the most likely way is that some “killer app” offers mesh networking as an option, taking advantage of its ability to (a) work where internet access may not be available, (b) transmit data faster than current methods of connection for sharing larger files such as video, and / or (c) preserve anonymity, since peer-to-peer traffic can’t be collected as easily by government or other agencies. It looks like this is what Open Garden is currently trying to do with Firechat.

I haven’t spent much time thinking about the problem, but I wonder what other applications would benefit from mesh networking’s advantages enough such that they can get it introduced to the public consciousness.

Thanks for reading! Join the discussion on Hacker News. You may also want to follow me on Twitter.

I would pay for Twitter. In fact, I would prefer to pay for Twitter.

Let’s talk about Twitter and Facebook. Compare their stock graphs, and in the lower-right of that view, check out the steady drumbeat of stories about Twitter’s stalled user growth.

I discovered Twitter a few years ago, took a break of almost exactly one year in 2013, and have been using it more and more over the past six months.

By contrast, I removed much of my personal information from Facebook a year or so ago, haven’t logged on in a while now, and I increasingly think that it’s junk food: a fun one-off experience that for many users simply produces addiction and unhappiness. That’s what I noticed in my interactions, and that’s why I quit. (By the way, if significant portions of your userbase are capable of becoming addicted to your product, consider that it may not be fully ethical to produce it. But that’s another blog post.)

Anyway, I’ve never had that problem with Twitter. I think that’s for a few reasons, all of which are probably not great for continued revenue growth:

  • Most people don’t understand how to use Twitter or what it’s for. Bad for user growth, good for users because it places certain constraints that make the bar for content production a little higher.
  • No “friending” paradigm. It doesn’t make me self-conscious to follow someone else’s timeline. Or to unfollow it. There isn’t as much social baggage. This makes the network less “sticky”.
  • Content sharing and curation is a more thoughtful exercise. On Facebook, you “share” content; on Twitter, you say it again. There’s something higher-stakes, to me, about “re-tweeting” to your followers. And you can’t “like” something on Twitter, but you can say “this is my favorite!”. Also reduces stickiness.
  • At least in my timeline, it’s most about sharing intellectual and professional content, rather than personal stuff. Facebook is the epitome of “comparing your inside to someone else’s outside”, i.e. having to measure your internal sturm und drang against a cherry-picked list of other people’s usually happy moments. But almost everyone has good personal stuff to share.

The ads in my Twitter timeline actually don’t bother me; many of them have been surprisingly relevant or interesting, in fact, unlike Facebook ads. So if advertisers are willing to pay enough to keep the company profitable, great. But it worries me that they might demand changes to make the company more like Facebook. If you doubt this, check out the new Twitter profiles and see if the format is familiar.

I would love the option to pay to ensure Twitter’s continued existence in a usable form (though of course there’s probably nothing to fret about quite yet). It’s kind of the paradox of free services in general, like GMail, or Wikipedia. Do I like getting stuff for free? Sure[1]. But if I paid for Twitter, perhaps they would be more accountable to me and not to advertisers.

[1] I actually do make a monthly donation to Wikipedia in the same amount as my Netflix bill; I highly recommend this if you can afford it.