jump to navigation

On data and content and learning November 21, 2013

Posted by ajackl in Educational Technology, Enterprise Architecture, Leadership.
Tags:
add a comment

My company was recently acquired by Houghton MIfflin Harcourt.   We really looked before we lept into the chasm of joining a large company like this.  The thing that tipped me over the edge was really two things: my experience of the executive team as being people committed to thought leadership and transforming their 180 year old company, and the company’s goal of creating passionate learners.

Now I am their lead data scientist and their VIce-President of Technology Strategy.  I am slowly coming to grips with the areas where I can lead and have a deep understanding – enterprise education data, platform architecture and design in the education space, system and data integration, mapping, assessment results data, and standards- and with the areas where I need to truly understand and learn from my new colleagues: content, assessment design, pedagogy, and content design and delivery.

I am going to post my thoughts here, and look forward to generating dialogues on these topics.

Advertisement

Identity: Starting with the Basics August 16, 2012

Posted by ajackl in Educational Technology, Enterprise Architecture, Standards.
Tags: ,
1 comment so far

As I have danced with the issue of identity management in education over the years  it has amazed me the degree to which it is an amazingly complicated and subtle issue.  It seems obvious at first glance but quickly becomes obscured in a cloud of issues.  So, to begin this series, I will talk about the use case of identification.   What is it, what kind of identifiers can we use, and what are we identifying exactly.  Let me start by stating the problem space “identity management” needs to handle.

In the non-virtual world we have various ways of identifying.  I will classify them into two types: people, places and things we know, and people, places and things we do not yet know.  For ease of writing I will focus on people but this conversation will go almost identically in talking about places and things.

We identify people we know by markers we pull from memory.  We know what their facial features look like, how they move, their voice, their way of speaking, how they dress and present themselves, and sometimes we are prepped with those memories by the context and location we are interacting in.  If we meet someone “out of context” we will sometimes take longer to recognize them.   I once knew a young lady very well who always wore red-ALWAYS.  It had become a prime association for me with her.  One day I met her walking down the street and she was in a yellow blouse and a brown skirt and I literally did not recognize her until she smiled and addressed me.  These combinations of factors often become so ingrained we are not conscious that we are processing individual attributes and factors but rather we just “recognize” people, places, and things.

We identify people we don’t know very differently.   We also use outward attributes such as appearance, clothing, location, and context to “guess” and make assumptions about people we don’t know.  This primarily has an impact on trust.  To what degree do you trust a stranger.  If you are not revealing anything personal, or financial or anything that could be “used against you”, and they do not appear to be hostile, then trust does not matter.  In those cases you spend very little time processing the “identity”.  However that changes the moment you need to ask them to do something or they ask or demand that you do something.  Suddenly you begin actively processing their identity.  There are many strategies.

You  may trust them more because they look, sound, and dress like you do.  If they clearly come from your culture or social class.  If they are the same gender or race or NOT the same gender or race.  I am not saying any of these as political statements but rather as pointers to the human condition.   The context is very decisive in these kinds of instant assessments: if you bump into someone in a bad section of run down urban area you will go through a different assessment process than you meeting someone who looks just like you at a best friend’s party.

You may ask them for their name. You may ask them what they do, what is their role.  You may ask them what they need from you.  If it becomes formal you may ask to see their paperwork- their passport, or driver’s license, or work ID, or their warrant.  You may call someone you know who they claim to know, or you may call their boss or their organization.  You may call the police or some other third party authority to ensure they are who they say they are and you can trust them enough to interact with them.

All these escalations are a function of how intrusive or private the request is and how comfortable you feel receiving or giving information or services to this person.  There are no fixed rules.  Every person has their own thresholds around privacy, generosity, trust, and safety.

If we take this then into the virtual world we have the same issues but we do not (at least in many cases) have the same visual, auditory, and contextual triggers and data to use as in the real world.  So we must find replacements or stand-ins for those.

In the virtual world we use four primary ways of “recognizing” people:

  1.  Usernames inside a particular context.  Examples of this are handles in a multi-player on-line game, Skype, Google Talk, a LinkedIn forum, a bulletin board.
  2. Email addresses
  3. Credit cards  These are  primarily person to application identification to allow access or approve execution of a service- like send this person this book, or allow this person to access this site , or add this person to pay this bill. This model can be and often is linked to number four below.
  4. Authentication through log in utilizing some kinds of credentials.

These are the “faces” and “voices” of the virtual world.  As we begin to move more and more into rich media collaboration and communication it may more and more include faces, voices and bio-metrics as those but as of yet those are rare indeed.

So this post is designed to introduce the problem. In my next post I will discuss the actual structures used to identify things in this virtual work: GUIDs (Global  Unique Identifiers), Dot Notation taxonomy-based identifiers, URIs in all their incarnations, locally-assigned identifiers attached to records,  content-based derived identifiers, and paradata -based identifiers.  These make up the “formal paper work”, the passports, the drivers’ licenses, of the virtual world.

I will probably start in the next post to also talk about third-party authorities (ICANN, DNS servers, credit card companies, social network sites, and certificate issuers) but that may be post number three.

Please comment, ask questions, clarify, disagree, elucidate as you will.  I will post again in a few weeks.

Identity Management in Education July 28, 2012

Posted by ajackl in Education, Enterprise Architecture, Schools Interoperability Framework, Standards.
4 comments

Over the next few months I am going to be attacking the issue of Identity Management in education.  As I  started framing this I was struck by the large number of different approaches and facets to this topic.  So I am going to post about three times a month or so and I will start by laying out in this post the topics I might cover.    Hopefully this will encourage some dialogue and give people a chance to react, ask questions and clarify anything that might be critical or interesting.

One set of questions my friend Wayne Ostler, from Pearson, asked me is:

  • What is this blog series trying to communicate?
  • Is it just wanting to inform readers about the issues/problem space?
  • Is this blog trying to suggest best practices?
  • Is this blog trying to set the stage for “standards”?

I am not sure… I think my intent is to explore this topic in a public space and use the conversation to inform the standards and to generate some best practices.

So here is a list of topics that  I hope to refine into a reasonable set of topics over the upcoming weeks:

Privacy: FERPA, HIPPA, record-level access, identifying data access, notification , disclosure, and consent
Security: encryption, authentication
Access: authorization, small-cell-size, drill through
Use of Social Security Number and other Personally-Identifiable-Information (PII) issues
Government mandates or funding:  “ARRA calling for cross-state, cross-segment data trails” (the government has many heads- not all of them realize what the implications are of what they are asking)

Types of Identifiers:  GUIDS, dot notation, URIs, content-based identifiers, biometrics

Person Identity vs. Record Identity:  Often people confuse these two and it is a critical issue when dealing with longitudinal data systems and identity.

Single-Sign-On (SSO) and Federated Identity

The dimensions of Identity Management: horizontal, vertical, cross-organizational, and time.

Building systems to manage identity in all its dimensions

The data elements needed to establish identity

Resources

Please let me know if there are topic areas I should include that I missed.  I look forward to exploring this topic!    Thank you!

 

Standards: Why and What? November 11, 2009

Posted by ajackl in Department of Education, Education, Enterprise Architecture, Standards.
Tags: ,
add a comment

My good friend Georgianna asked me a series of question in her comments to my post. They were great questions so I decided to blog on them!!!!:

>I find this conversation interesting and wonder if you could discuss the “why” of the standards?

This is an interesting question which I am daunted at the prospect of answering in a real way as opposed to giving a glib answer. Philip Howard wrote a post recently that speaks to some of the data reasons for a common data standard or data model.

I think there are some important considerations that make standards important. The following five points were identified by the State Chief School Officers as critical to the nation’s data needs (thanks to Chris Lohse, Director of Data and Research for his articulation of these). These are in my opinion the reasons why we need data standards.

1. Liquidity – Data liquidity is by far the most discussed challenge around education data. Today, the term has gained much more traction in the context of healthcare and patient records, but the fundamental concept as applied to education is that a student’s data should move with him or her and should not be bound to a particular school information system or district data system. As one healthcare commentator noted, it is after all, the individual’s data.

2. Comparability – We struggle greatly in education with developing common definitions for even the most straightforward of education terms. During a workshop in Toledo, OH, participants at six different working tables were asked to write their definition of ‘high school graduation rate.’ Participants were then asked to pass their definition to the next working table and have the participants at that table interpret the new definition just handed to them. The results demonstrated 1) nobody had the same definition of ‘high school graduate rate’ and 2) nobody agreed with the definition that was shared with them. Until common definitions for even our most basic educational outcomes are defined, we will be greatly limited in our ability to actually understand what is happening among all students across the country – and even across a state or school district.

3. Relevance – Schools and districts collect swaths of education data but much of what is collected has limited value to teachers, parents, students, and school leaders. More often, the data are merely tracking outputs, rather than an actual change in status or condition. For example, under No Child Left Behind, many saw limited value in knowing whether teachers were highly qualified according to certifications, tenure, and degree attainment. Rather, a more relevant outcome and metric would be a gauge of whether teachers were effective in terms of student growth.

4. Timeliness – Discovering that a particular student is behind their growth and learning trajectory at the end of a school year, with little to no time to change course, is of limited value. Instead collection, analysis, and reporting information about students, teachers, schools, school leaders, and districts should take place on a time schedule that allows for change of course and performance improvement. Data and analysis provided beyond the point when corrective action can be taken is, at best, useless. At its worst, it is irresponsible.

5. Accuracy – The strategy and analysis emerging from good information is only as good as the underlying data. Increasingly, self-validating data collection tools are improving data accuracy. Put simply, when relevant data are presented to teachers and other education stakeholders in a timely, coherent way, those stakeholders begin to ensure the quality of the data inputs.

>Are they to define or align expectations in US education?

This is a key thing for us to remember- there are different kinds of standards. There is work ongoing with 48 states to create a comparable, standardized set of student expectations. These are academic content standards. I am primarily talking about data standards.

These are more about defining data elements, their data type, their length, their description, their relationships with other data elements. For example – Student First Name, Course ID, Enrollment Status, etc. I would call these “data standards”.

There is a new effort coming under way which I will be involved with and documenting closely to set data standards in the country. This is being done in partnership between USED, CCSSO, SHEEO

>If so, in what realm – student expectations, teacher expectations, system expectations?

Yes – the direction is to both support next generation learners and teacher development. System expectations are directly correlated to that. I don’t have space right now to discuss this but it is happening.

Another question that came to me from a discussion with Jean around some pilot work in healthcare – these pilots have created a hospital in which doctors aren’t compensated in a fee for service format, but rather by salary with bonus incentives around patient outcomes.

I began to wonder if there is work going on to define standard desired outcomes for school systems and if so, if there are any areas where teams of teachers and administrators are given an incentive bonus for particular student outcomes?

This is a huge issue. I won’t attack it in this post and much has been written about it, but it is clear that teacher effectiveness and accountability is going to be massively focused on and it is- I believe- a general consensus that some kind of growth model is necessary to do any kind of reasonable analysis of effectiveness of students, educators or systems in education.

Enough for today- more later. Thanks for the questions!

You say Data Standards, I say Data Standards… October 24, 2009

Posted by ajackl in Department of Education, Education, Enterprise Architecture, Schools Interoperability Framework, SIF, Standards, Technical.
Tags: ,
5 comments

How many different versions are there of data standards? There is a lot of talk nowadays about data standards in the education space and the need for data standards. The Gates Foundation have become standard bearers of that cause (Sorry- couldn’t help myself!). Secretary Duncan has said it is one of the key parts to education reform. There is a great cry and hue for National Data Standards. The Council of Chief State School Officers (CCSSO) and the State Higher Education Executive Officers (SHEEO) both are pushing for state-lead leadership in this domain and emphasizing its importance, and CCSSO is driving the national academic content standards on bahalf of the Chiefs and in partnership with all the majopr players. I couldn’t agree more with this emphasis. I have given much of my recent life to empower and support data standards of all kinds. However, I find myself in many conversations where people use the term “data standards” and I find myself quoting Inigo Montoya from _The Princess Bride_:

“You keep using that word. I do not think it means what you think it means!”

There are many kinds of standards. To begin a conversation for, ahem, standardizing on some we need to start by determining what kinds of standards we are talking about. In this post I will start listing some standards. I started thinking about this topic newly when my friend BethAnn from Virginia’s Department of Education disturbed our post-conference-session drinks with the question of “What do we mean by standards?”.

There are many ways to think about that. One way is to think about “de facto” standards versus “de jure” standards vs specifications. I think it is important to recognize which ones we are talking about.

“De jure” standards are standards set by law, by some authoritative body that has a mandate and the power to say how it goes. In the National Data Standards movement we need to be cautious that we are clear that we are not creating a mandate but an agreement by consensus. If people perceive that the United States Department of Education (USED) is telling them how they need to do their data systems, many will dig their heels in and it differently only because they can. On the other hand, if USED does not take a stand and display leadership then many will not take it seriously. It is a difficult balance we will need to strike. The “http” and IEEE standards are good examples of technical “De Jure” standards as are the various state’s academic content standards.

In fact “De facto” standards are what we need- standards that arise out of use and agreement as THE way to do something not out of an authority but rather out of consensus. That, unfortunately, will require a great deal of communication and more bandwidth between users, state education agencies (SEAs), and local education agencies (LEAs), and vendors than exists right now. The Schools Interoperability Framework (SIF) specification has become in fact, in many ways, a “De Facto standards for interoperability in the k12 space, as the Postsecondary Electronics Standard Council (PESC) has become for post-secondary education. The NCES Handbooks are another example of this kind of standard.

Specifications are “a” standard way to do something rather than “the” way. SIF, PESC, and Dublin Core are all examples of specifications. These are easier to pull together but less likely to get adoption because of the lack of authority. This gets blurry when some states start to mandate SIF for example and thus essentially turn the specification into a “De Jure” standard.

So that is one lens to view standards through. I wonder what the dictionary has to say… according to Merriam-Webster Online 2009:

Etymology: Middle English, from Anglo-French estandard banner, standard, of Germanic origin; akin to Old English standan to stand and probably to Old High German hart hard
Date: 12th century

Interesting – so we need to stand hard.

1 : a conspicuous object (as a banner) formerly carried at the top of a pole and used to mark a rallying point especially in battle or to serve as an emblem

It could be argued that this conversation is a rallying point in the battle for education reform!

2 a : a long narrow tapering flag that is personal to an individual or corporation and bears heraldic devices b : the personal flag of the head of a state or of a member of a royal family c : an organization flag carried by a mounted or motorized military unit d : banner 1

Unfortunately some of the players jockeying to control the National Standards Movement have this in mind rather than the good of education in this country! No, dear reader, of course I am not talking about you!

3 : something established by authority, custom, or general consent as a model or example : criterion
4 : something set up and established by authority as a rule for the measure of quantity, weight, extent, value, or quality

Pretty clear.

5 a : the fineness and legally fixed weight of the metal used in coins b : the basis of value in a monetary system

Nope. Not this one.

6 : a structure built for or serving as a base or support

This is my favorite- we are building the national data standards as a structure built for or serving as a base or support.

7 a : a shrub or herb grown with an erect main stem so that it forms or resembles a tree b : a fruit tree grafted on a stock that does not induce dwarfing.
8 a : the large odd upper petal of a papilionaceous flower (as of the pea) b : one of the three inner usually erect and incurved petals of an iris

Ahem. I really didn’t know what to write about this one! 🙂 We certainly don’t want to induce dwarfing in our education system.

9 : a musical composition (as a song) that has become a part of the standard repertoire

We can all sing along!!!

Okay. That was fun. But that isn’t even the end of it. There are also different domains of standards.
SIF is an interoperability standard – it creates a blueprint describing how educational data can be moved from one system to another. PESC is the same except its scope is postsecondary, and SIF’s scope is K12.

The NCES Handbooks and the National Education Data Model (NEDM) are content standards. Although NEDM is more of an aggregation of all the standards and data sets out there than a standard itself.

The standards about what a student must know, or the assessment benchmarks that a student must hit are curriculum or academic content standards.

There is a lot of confusion about these different domains and when we are speaking in this national dialogue it is important that we are clear which domain, type and definition we are using.

It is my commitment that we will do this – and that what we build together as we start to build the National Data Standards is clear and that our approach is, well, standardized.

More later. Stay tuned.. same standard time, same standard channel.

Vertical vs. Longitudinal Reporting February 25, 2009

Posted by ajackl in Educational Technology, Enterprise Architecture.
Tags: ,
2 comments

Someone asked me a basic question which, at first, I thought was obvious and then upon asking around discovered was not so obvious.     In the State enterprise system conversation what is the difference between “vertical reporting” and “longitudinal reporting”?

The answer is simple and complex:  time and space.

First, space: “vertical reporting” refers to the movement of data between organizations at different levels of hierarchy in their area.  For instance, many schools belong to a district, many districts belong to a regional service center, and many regional service centers belong to a state (some states skip the regionals).  Vertical reporting is the movement of data up that chain of organizations:  school data go UP to district which transmits them UP to the state.  Sometimes people will talk about “vertical interoperability” which means data can go both ways.

Second, time: “Longitudinal reporting” refers to the reporting, acquisition, and dispaly of data that have a time element to them.  It is often represented by graphs.  For instsance the change in enrollment over the last ten years would be an example of longitudinal reporting. Longitudinal reporting is the reporting of data with a time element attached and used as a dimension.

When people refer to longitudinal data systems (as in LDS Grants) they are referring to systems that almost always have a vertical and a longitudinal component to them.

SIF doesn’t work… or does it? February 25, 2009

Posted by ajackl in Educational Technology, Enterprise Architecture, Leadership, Management, SIF, The Three Laws of Performance.
Tags: , , , ,
2 comments

I have been observing an interesting phenomonon at play around the Schools Interoperability Specification.   It has to do with the First Law of Performance from a book I am reading right now (http://www.threelawsofperformance.com/).  The First Law states that people’s actions are correlated to the way the world occurs to them.

At one level that is a no brainer.  Of course that would be true.  But we don’t act that way.  We act as if the way it occurs for us is the truth.  An example of this came up at the National Education Data Conference in Seattle last week.    Someone mentioned to me that a panel of people had spoken to all the states who had recieved longitudinal data grants from the government and had shared how SIF didn’t work.  They spoke about it like an “of course”.  Other states though were walking around talking about how SIF was revolutionizing their data quality and data collections.  How could they both be true.   I have heard these extreme positions multiple times.

What became clear to me is that people were speaking, and complaining, based on the way the world occurred to them.  It isn’t that they were “wrong”, though I was apt to leap to that conclusion,  they were just speaking their “TRUTH”.  What is so is that SIF is working in hundreds and hundreds of districts in a horizontal (which is to say local to a school or district) or “classic” deployment, and is working statewide in a few states in a vertical manner, and half a dozen other states are working on it.  So why the bad press?

I say it is because the problem that SIF is brought in to solve is complicated.    Building enterprise architectures out into deployments that scale multiple levels of organizations (school, district, regional, state),  broad breadth (all the districts, all the schools) , and heterogenous applications (Student Information Systems of multiple types, Transcript brokers, state data warehouses, data collection tools, etc.) is complicated stuff.  The data flow inside any one of these components is complicated enough.  Managing the data flow through and around them all is hideously complicated.  Thus the issue.

Most states are not approaching the problem with an appropriate respect for that complexity.  They buy a product (like an off-the-shelf “data warehouse” or similar product)  and expect that to solve the problem without really mapping out their issues, use cases, data architecture and process flows and then putting together a system that works.    Then when the system falls inward on itself or doesn’t hit its milestones it becomes “SIF doesn’t work”.  The truth is  enterprise system design is hard, and almost impossible to succeed at in a political, consensus-driven environment.    There are so many points of failure and SIF- in automating the processes- reveals those breakdowns and issues, and, as with most messengers, often loses its head.

When you want an assessment of something make sure you are asking someone who understands those types of problems and has a track record with them.   The person who has never succeeeded may just not know what it takes, and then they will cry to the world “Don’t do X”.  If we can understand the root cause of the failure suddenly the world will appear a different way and then our actions will be correlated to that new view.

Now, telling who the real experts are and who the savvy sales people who memorize the jargon du jour… that is another problem entirely!

The path forward- taking strides not small steps! January 22, 2009

Posted by ajackl in Education, Educational Technology, Enterprise Architecture, Schools Interoperability Framework, SIF.
Tags: , , ,
2 comments

I have been thinking about what we will use the economic stimulus package for.  Particularly the parts impacting education.

I am going to not address tools that would be purchased singly by a teacher or school, but, rather, enterprise solutions- with economies of scale- that would be implemented at the district or state level.

Here is my initial list.   I hope you are listening Arne Duncan!!! 🙂

1. Flexible, SIF-enabled, unit-record-level(student, staff, course/section, building, district),  data quality-enforcing data collection systems.

2.  Semantically-tagged, multi-purpose, longitudinal, flexibly periodic, data storage

3. Enterprise Directory-enabled Portal solutions that provide authorization down to the parent, student, teacher level and provide security for both access to applications and resources but also constrict viewing parameters on reports.   Centralized management and indentity services but decentralized control

4. Inside these portal solutions Communities of Practice (data providers, teachers, families, program providors),  with collaboration and file sharing capabilities as well as access to reports designed specifically for them, and workflow tailored for them.

5. Learning Standards Repository and National Data Model and NCES Handbooks all available in referencible form and allowing free access to the date through a variety of APIs.

6.  Working with SIF and PESC and others to construct Transcript standards  and templates for the whole country

7. Mature the SCED specification to include elementary school and more metadata.

8. Mature the SIF specification to provide more value and out of the box interoperability to systems all through pk12.

9.  Provide Best Practices for maturing systems that are not driven by a single solution-providor but by healthy, standards-linked functionality.