 So, welcome to this presentation on transforming interactions between government and communities through the use of social media. I'm Dr. Pat Franks, and I do lead a research project through InterPARES Trust that studies social media and trust in government. And Lacey Ryan Banks, we'll call her Ryan, is a graduate research assistant working on the project with me. And she's doing some work on specific cities, local governments. So, she's going to be giving you some real world examples of what we're looking at here. And this is only a piece of the project. When I get to another point, I'll explain to you how one of the other instructors here at San Jose is involved and where. So, our talking points are just really three. What it means to trust, social media and power and identity, the relationships and local government and social media. And then it'll be summarized in just a few key points. So, what is trust? According to InterPARES Trust, the research agenda that we're following, there is a terminology database. Trust is the faith or belief in a person or a thing with confidence of some action or the expectation of the safety of the thing. So, really what we're saying is I believe you're going to do what I expect that you will do, right? And that it will be good for me. So, that's what trust is here. Trust is associated with an agent is in a financial trust or some type of trustworthy system. And then we get into a term there called trustworthiness, where we're looking at the accuracy, the reliability, the authenticity of a record. So, there are a lot of ways to look at a definition of trust. But the way I'm using it here and the way we're using it in our study is I believe you're going to do something that is going to be good for me, my benefit. So then, what is trust in government? Well, according to Levi and Stoker, trust in government is an evaluation of whether or not political authorities and institutions are performing in accordance with normative expectations held by the public. So, if we take a look at this slide, you'll see that the first principle in the code for federal employees. Starts with the statement, public service is a public trust. And if you were trying to determine trust in government, you might ask a research question like, do you have confidence in the local or federal or state government to do what is right? And then you would provide some type of a like or stale that they could respond to perhaps saying never some of the time, about half of the time. Most of the time are almost always, definitely not almost always right now. As you can see on this slide, perceptions of trust in government according to the Pew Research Institute is at an all-time low. It went from 73% in the Johnson administration. That was right after the Kennedy years when there was a lot of enthusiasm. And even though there was that horrible event and President Kennedy lost his life, there was still this feeling that we could do almost anything. So, trust was pretty high at 73%, but it fell dramatically. And there were a lot of events, of course, that occurred of Vietnam War, for example. But where we are now, you see on this, is like 19%. And this is perception of trust in the government. And this was the end of 2013. It may even be lower now if you've been watching the news or checking into your social media accounts. Perception of trust in government in Canada is a little bit better. This was measured by an ECOS Research Associates. And the question that was asked here is, how much do you trust the government in Ottawa or Washington? Because they did also ask the US citizens to do what is right. And if we wanted a one-sentence summary of what the polls told us about Canadian democracy in 2013, it's reported that the statement would be, we're losing faith. And I would say that, of course, it's the same thing in the United States. Now, there are some countries worldwide that are not experiencing this decline, this dramatic decline in trust. But our two countries certainly are. Now, when we're looking at trust and we're trying to tie it to social media, what we're looking at is social media is a source of information. And so the second thing we're trying to determine is, so really, what source of information do you go to? In order to get the breaking news, to get your regular news, to check up on what you've heard, if you want to confirm something that you've heard that should be in the news. And in this case, there's a formula for trust, which means that you've got authority and helpfulness and intimacy. Those are all the good things. But that's divided by self-promotion. I see this self-promotion happening a lot on mayoral sites. And I was just noticing it today, the mayor of New York City not standing or supporting the police department. Not that I think he should have, but I've noticed this on many other sites that we visited. One department may be down as far as citizen trust. Well, another one is higher, and it's usually when they are agreeing or servicing, providing good service to the community. So it's kind of a very complicated situation to be able to understand. But the important thing here is that social media now is at a 47% trust level. That means that 47% of the people would look to social media and believe what they say as far as is included in the social media. So they'd look at Twitter and say, I do believe that that's correct, or I do believe what I'm reading on Facebook and so on. Traditional media is at 65%, but you'll see that online search engines now is equal. So what we're learning from there then, if you wanted to apply it to your own organizations, is that traditional media is expensive, and it may not be something that you need to use in order to reach your patrons, your customers, your citizens. Because the same number would be able to find that information on the website, and it would cost the organization that's trying to get that message out a lot less. This is very important, of course, for local, state, federal government. So social media, trust in social media is rising. It's been rising over the last few years. Trust in traditional media has been declining. Now, when we look at both trust in government and trust in social media, trying to see if there's some kind of a relationship, if we do a better job at social media, will that increase trust? We look to what the government is doing now, and if you have engaged with your government electronically, perhaps paying your taxes or voting or purchasing something, I know I pay for my driver's license that way. This is all e-government. The government is using the technology, but it's to support government operations, to engage citizens, to provide services. But it's kind of that one-way street. You are in there. You are performing some actions. You're entering into transaction, but it really isn't there to get you to collaborate with government. What it's there for is to provide you with a service. That service should actually be cheaper for the local government, state government, federal government to provide because they're using technology. But what we're interested in is how can we go beyond just if you were a government organization, how could you go beyond e-government into the e-governance? The e-governance is a little different. This is where citizens should be able to cooperate, collaborate, communicate with the government and to actually make changes in the way the government operates. And we're seeing a lot of that now if you've been watching TV with the protests. It's one way for individuals to attempt to make a change in a system that they believe needs change. What we're looking at is how can social media be a part of it? Is social media a part of this now? So when President Obama came into office in 2009, he declared that he wanted an open and transparent government. And he told his agencies that they should start using social media in order to share information with the citizens. So that was the beginning of in the United States openness and transparency on part of the government, at least as a philosophy. So social media was used. And if it's used correctly, public trust should be increased. If it's not perceived as being used correctly, if it's just being used as a media campaign, for example, to try to tell someone this is correct when it's not, it's not going to work. And I've got a couple of examples of that for you too. But what we're believing, what we believe in our research here is that openness and trust through the use of social media can increase public trust. And if you follow that arrow across, if public trust is increased, then social capital, social capital is the feeling that an individual's interests are met. You know, if your government fills a pothole in front of your yard, because in front of your house, because you call and say, I've got a pothole there and you know, it's not good on my car. Well, then your individual interest has been met, and so you feel really good about that. So there's good feeling there that can equate to better thought about your government and then better cooperation on your side. Maybe you'll vote to increase taxes next time so they could get a new truck to do filling these potholes. But also to share an interest and then to common interests where you have groups that are actually working together to do something. So public trust, though, can also be decreased depending on how social media may be used. As I said, I'll show you an example of that. But in any case, whether for good or for ill, policymaking can be affected. That's what we're trying to look at. We're trying to study how this will happen and for public administrators eventually to come up with guidelines that will help them understand how to use this in a good way to increase this involvement, this interaction with their citizens so that everybody can have their needs met. Now the next part of this is about social media and power and identity. And we actually have some impressions of the government already, right? How many of you have seen this before or heard it? I'm from the government. I'm here to help, right? And you roll your eyes sometimes when you hear that. But that's a cliche. But there are some and I'm using the police department only because my father was part-time police officer. Oh my gosh, in a very small town. And so I have a fondness for visiting police departments, but I'm not always saying they're right. So, but in this case, I found this one and Brimfield, Ohio is a town of 10,000 people. And yet back in 2013, they had 88,000 Facebook likes. I mean, so they were doing something right with their social media. And it's of course not all because they were doing social media and they were doing it right. It's also because that police department was doing something right. And then they used social media as a means of communication to let the community know what they were doing, what they can do, what kind of services they can provide and to listen when the citizens engage with them in Facebook and social media. In order to get that reciprocal agreement going, although of course not a signed contract, but this understanding that this trust that the police department in this case would do what's right for the citizen that was asking for assistance. Now in this case, of course, it's one of these pictures where the police department helps unfortunate disadvantaged youth go shopping, get ready for school that type of thing. So that's always a good thing. Remember that number, 88,000. All right. Grimfield police use all kinds of devices. They make sure that their social media can be adapted to mobile devices. They use Facebook and they use Twitter really beautifully. And they do have the app that you can download so that you can follow them. And if you would look on this slide, what you can see is by now they're up to something 146,359 likes. Now that I think was in the spring. So that was maybe nine months or so after the 88,000. They've moved up to 146. The only city that has more interaction on Facebook than they do with citizens is New York City. New York City is so much larger. Remember, there's only 10,000 citizens in this town. So on their site, I went to look to see what they were doing right. You know, why do people keep coming back? And I saw this picture of the dog the first time I visited. I was really sad because I didn't know why that dog was behind the chain link fence. But as I read, of course, I found out the police chief found him in the morning. And he says, please help us with this young puppy wants to find his way home. And so they put out a call and they put out the picture. And they've got that little guy there that they're taking care of for a citizen who has lost that puppy to be able to come and get it. So not like in a cattle or something when they're picked up and nobody knows where they are here using social media to let people know. So what I got out of this right down below where it says involved is somebody saw this and said whatever happened to the Golden Retriever. So apparently they've done this before and also appreciated right before that is, you know, this is really awesome that you do that. So you're seeing there's a service for someone. Others appreciate that and know that if their animal happened to be loose and was picked up. I'm going to dog catch her in this case, though, the police, they would be notified through social media on the right hand side. This was from today. This town is not unaffected by the events that are going on from Ferguson and from now New York City. So the dialogue is about police wearing cameras. And this is a picture of a policeman wearing a camera. And I didn't read the article underneath, but what interested me was one of the tweets way down at the or I should say post down at the bottom kind of fell in with my interest in social media. It's about keeping track of it, right? And somebody says, you know, this is another concern that we're not even talking about if you do wear these cameras then. How do you retain them? Because you do have a data retention rules that have to be followed. And also freedom of information request. There's a freedom of information act. So these cameras then would videotape that tape. Can you tell? I'm old videotape. They would record. The digital information would be recorded on servers. You would have to have rules now as to how long you would keep that before you would reuse wherever they happen to be stored or before you would destroy them. You'd have to make sure they were available for individuals that want them through freedom of information and are entitled to them. And then you have to build on the cost because all of this costs something. So police department budget is going to be affected as well. So anything that we're talking about really affects the entire organization. Now this for my town before I leave it, at Brimfield, December 3rd, that's today. They're now up to 174,991 likes. And I would just say visit it. You want to see what people are saying even from other communities that don't live in this town, that they get services met too. And that they just really appreciate everything that's being done. However, there's another side to this too. Our poor eagle is saying really now I've given you all the good news. You still trust the government. Does everybody really trust the government? Well, if you ask the Native Americans, they might say, oh, not really. Look what happened to us, right? So now we have the really sad side of this. We have the Ferguson department, the pictures were taken. And what happens today is that you can't erase what's out there. We can say for retention, disposition, our organization can remove whatever responsible for. But somebody else has the picture somewhere. And so unfortunately, these pictures are out there and they're making an impact on citizens. And when we talk about trust in police or trust in mayor or trust in city council, images are so powerful. In this case here, not only images, but we have someone who used his Facebook account to threaten the police officer in this case. But what is really sad is he not only said, you know, now we have to shoot the police officer. But if we're unable to locate him, then let's shoot the wife and the children. And so we see the anger and frustration being built and mounted. And the police, again, trying to keep some kind of order if possible. And then you have all kinds of people just chiming in on their own Twitter feeds. This is one. And this is where Charles Barkley, you know, basketball player, right, said that maybe profiling is wrong, but maybe sometimes it's not. So he was trying to get into a conversation, a dialogue, which didn't go well because then people were upset with him on social media. So his last tweet or not tweet, his last statement today to the press was I'll never do social media. I don't want to talk to those idiots. So here we're getting someone very upset with being, I guess, bullied for his comments on social media. Now, I know this is a lot of information that are just examples. But I think what you're seeing is the contrast between services that are assisting the citizens, the way that Brimfield used social media in order to get people to cooperate and to collaborate and to appreciate what they're doing. And then on the other side of it, how it can be used in order to, I think, make things worse sometimes. Now, some of this information, if you were looking at it, you'd say, why do they keep that up on their site anyway? Because people are there and they're upset and they're posting things that maybe would incite others. But really, if you're an elected official, you've got to be careful even about removing things. This example I thought was really, really directed to the point. It was Indiana Governor Mike Pence. And when we talk about trust, this was broken trust. He really didn't even do anything wrong. Somebody who worked for him did, but he was the one who felt the brunt of it. He had an idea about how the Supreme Court should rule on a case before it. And what he did was express his personal opinion, which he might not have done if he had thought about it a little bit in advance. When he expressed that opinion, the citizens that were following him were very upset. And within 24 hours, there were 1,000 comments, and not many of them were in his favor. And so somebody on his staff decided to delete the comments. And that was not a good thing. Because what quickly came up was another Facebook page, and it was called Spencer Ship. And the people who started that Facebook page invited everybody else that had their comments deleted from his real Facebook page to post again. And they did. So the resentment had mounted not only did they disagree with what he said in the first place, but then by removing all of their comments, they disagreed with that action. Unfortunately, the governor then he did apologize, but he came out and he said, we need a new improved policy on Facebook. So remember my circle where I'm saying feedback will then impact the policymaking. The only problem in this case is I read the policy and the policy was clear they should not have been removed. So it wasn't that what it should have been if this really was a new person that didn't know it really should have been training. So that's what was lacking in this case. Briefly, I wanted to show that we in the United States and Canada are not the only countries that are dealing with social media. Social media in the Arab world is influencing societal and cultural changes as well. And you can see here that social media is empowering people in these countries in order to understand where they fit in. And it does influence change in their community. And you'll see Saudi Arabia is probably the lowest of 32, but you go up to Kuwait with 52. And it's empowering Arab women, for example. So it's quite powerful if used properly. Additional questions related to an individual's sense of identity. Remember I said, you know, you identify that's what we saw with the people on one side or other side of any kind of controversy or issue is that they're identifying with that. In this case, people are learning in Lebanon and Egypt, for example, that use of social media increases their national identity. But in other cases, it increases their identity as a citizen of the globe, not just of their own country. And you see that United Arab Emirates and in Jordan. And then in some cases, it does enforce their regional identity, but even the religious identity, which I found surprising. I mentioned that I did have just a couple of just that line. I had a couple of sites here that affected this. And again, if you trust that what you see in social media is correct, then you would say these are correct. If you did not, of course you would say this must be made up by someone. And that's something that we have to be very careful with to are these official sites that we're looking at? Or are they sites that were made by someone else to look as if they were official? So even the United States government has a site that helps you identify which are official sites. Otherwise, you could be fooled. But in this case, this is an extreme where the founder of Russia's Facebook, and it really isn't Facebook. What is it called? It's a book. I'm not sure I'm saying it right, Svetlana. So maybe you know and I don't. But what they're saying here is that the government took over their social media. Go ahead, Svetlana. Do you know something about this? Yeah, I heard about it. Yeah, that he was kind of forced to give control. Not mostly for content, but mostly for people who post, who make posts. And then the special services can show away and attract the people who make some posts. So you're saying this really wasn't to control content? The way the headline is written is not correct? Yeah, I think so. Okay, yeah. And it's very difficult for those of us who are seeing these messages to understand what is true and what is not, even with what we see here. I see on the right-hand side, this was someone who was trying to get out a different message. This came from the Ukraine. And what they were trying to do is get out information about some people, I think, who were killed. Oh, I don't have that back there. What I wanted to end with though on this one was that I believe the site is up and being run independently now because I went back to it and it looks fine. And you could get in and become a member of it. So it doesn't seem as if it were censored as I thought it might have been. And I just tried that today. So I found this very interesting that we could get information through social media. Other countries have their own kinds of tools that they use. And other ways of, I think, either monitoring it or being able to use it to provide information. And I would like to say influence and not control. I spoke with someone from China and we were joking about social media. And she said, you know, you notice I don't categorize any of this. I said, why not? Because she said here in China, everything is under control. So she was kind of taking the other stance that in their country it's very much out of their ability to be free with what they want to communicate. I don't know. But what is interesting about the Interparez Trust research project that I'm on is that this is not my problem because we have people from the Ukraine and Russia both looking at these issues, social media from their own perspective. So we will get the perspective from the people in those countries. We have researchers all throughout Europe, South America, Asia and it's just unbelievable. So we can't wait to share information once we're all done with this. But now I want to turn this over to Ryan because she's going to talk to you about what we specifically are interested in. This is the part we're working on. It's local government social media. So Ryan, you take it away. Okay. If I could just have moderator privileges. Oh, you did. Yes. And I have to give it back to you because you went out. So I will do that right now. Thank you. So as we see here, there are 20 cities that we have decided to look at. 10 Canadian, 10 American. When I came on to this study, they've been working for a year. So all these cities are already picked. And when you came in, as you see, there's the list of the US cities. They're pretty fairly diverse. Of course, New York's the biggest one out of all of them, but they range. Most of them, I would say are mid-sized cities or excuse me, the most part. Yeah, most of them have at least 300,000 people with Riverside being the smallest. So fairly large cities, at least on the US side. I don't remember the data for the Canadian, but I would guess they're probably very similar. So one of the cities that I've been looking at is Raleigh. Raleigh is one of the major cities in North Carolina going to our next slide. We can see that it was founded in 1792. They are very big on banking and finance there, in addition to them being part of the research triangle, which includes Raleigh, Durham, and Chapel Hill. They're social media. Well, for their e-government, they've placed 10th in the 2013 Digital Cities Survey. And this year, they were a finalist for Best of the Web City Portal category. Now, if you actually go to their website, it's fairly bare bones. There doesn't seem to be a lot of data provided. However, they are very good about providing links to all of their online services. So for the most part, Raleigh seems to use social media to inform their citizens about the programs and services and events and stuff that they're doing. And they like to also promote things like their recreation and their parks and other stuff that will actually go over later. But as we can see from this page, this is what their social media page looks like. They actually do keep it fairly updated. As you can see, it was just last updated on October 30th. And from there, you can access all of the major city portals. So their Facebook page and whatnot. And at count in September, they had, let's see, 2,585 Facebook likes. So not a whole lot. Not a lot better with Twitter followers where they had 16,800. And then their mayor had like 1,500. So as you can see, City Portal is definitely what is used the most out of all of that. This is the main thing that we've been working on lately. One of the other researchers, in fact, I believe she is a professor for San Jose. She took all the data that was gathered from Twitter and analyzed it. She found out how many tweets they did. And then she ran a program to see what was positive versus what was negative. For most of the cities, positive, neutral, I'm sorry, was the most common category. So they didn't really say anything that was perceived as good or bad. For Raleigh, they're actually a really positive neutral city. They had one of the smallest negative analysis from the column there. There's ended up being an average of 4.9. And so it was just, that's actually pretty impressive, considering all the things that can go wrong for a city. And as you can see, their tweets tended to stay fairly consistent, though I noticed there was a change in May and August. I should go back and say what I have been doing is going through this analysis and finding out what was going on to cause these, we'll call them wild variations, though they're not very wild. But starting in February of this year, they got, they started going over 400, almost consistently except for whatever happened in June. August 2013 was the highest number of negatives for the city. And there's a very good reason for that. They enacted a law about not ceding the homeless. And as we've seen in Florida, people are not happy about that. Well, same thing happened in Raleigh. People were tweeting all the time about why would they enact this law. It was kind of crazy to them. Like, why would you charge $800 a day for permit to feed the homeless? People were very unhappy about that. They were also, something was also going on with trash collection and road work. That month, don't know what happened quickly, but yeah. And then in September actually turned around and became their month with the most neutral statements. And then going on, February had the most positive statements. And from what I was able to gather for that, a lot of it had to do with publics was coming in and building a new complex. So that also, they were also talking about other business and development conversion projects. And of course they got hit with what we like to term here as snowpocalypse. They got hit with the second part of that. And people were really impressed with their responses that they were putting out. So they were telling people about road closures and school closures. And they were staying really, really on top of telling people about the winter weather problems that were going on, which in getting what happened in Atlanta right before then, I think people really appreciated that. And another major thing that they were talking about was the fact that Google Fiber might be coming into town. And for anyone who lives in a major city, we're all waiting for this. So a lot of it was talking about how much better Google Fiber will be than Time Warner. So that's basically what is going on with the statistical analysis. It's just going through seeing what happens, where, why, and how that translates into citizens engaging with their government. So I'm going to give it back to Dr. Franks to talk about our key points. Thank you, Ryan. And I didn't intend to say this, but you had mentioned the snowpocalypse. And one of the events this fall was snow in Buffalo area. Here in the Northeast, we sit pretty hard. And just as you mentioned for Raleigh, the areas up here in Minneapolis too are using mobile apps for citizens to be able to report needs. So during times of a disaster like this, which can really be a disaster when you have so much snow all at once, but they're also being connected with the Internet of Things. So there's a lot of requesting, for example, plowing in certain areas. And the messages are, they're using GIS and they're actually sending transmission right through the government without having to call, without having to worry about anything. And they're mapping the locations of where all their trucks are so that they could better move them when they need them. And what they're saying is that there's a lot of interactivity with the citizens, and the citizens are benefiting from the investment in technology that the government has made. So those are the good things that you see that do increase the public trust, public attitude toward the government and they see that they can trust their government in times of emergency to be able to handle things in a way that is good for them. But as Ryan said, we have some key points and then we have time for questions and answers. Mainly what we're taking away from our study, and Ryan did just one of 20 cities. And in each city, we did not only their Twitter account and the Twitter accounts for the police chief, the mayor and the clerk, but we also did take a look at Facebook pages. We looked at all of the social media they were using. We looked at and are still looking at the policies. We're looking at the way they are maintaining records, whether or not they consider social media records that have to be maintained, if so, how they're doing, capturing them and retaining them. And what we're hoping to do at the end of this is identify two cities that are outstanding as far as service positive effects on the citizens in Canada and in the U.S. and to glean from that tips that we can then share with other public administrators so that they can understand how to use social media more effectively to be able to engage with their citizens in a positive manner. So it's been really interesting. So what we're saying is social media then is increasingly used by government to inform, interact with and influence citizens. And it's also being increasingly used by citizens so to inform, interact with and influence their government. So there's that mutual tension there. And the use of social media, the way we're using it now, think about the history books and think about the artifacts that we have from the past, you know, and the way we look at history and try to find out what happened before us. What we have is social media as artifacts or we will have them if we can retain them. And what we're seeing now being played out in Facebook and in Twitter and LinkedIn and all of these other social media, your Pinterest pictures there. This will be our history. This will be a record of what life was like at this time if we can find a way to preserve it. So this social media will have an impact on memories and identities and on our communities. So the research agenda again that we're involved in really wants to take a look at local, national, international networks of policies, procedures, regulations, standards and legislation concerning digital records that are entrusted to the internet. So not just the social media. We're looking at all kinds of digital information. And what we're trying to do is ensure that public trust can be grounded on evidence of good governance. We're trying to find a way to make it right where there are problems and that this will create for us, if we can do this right, a persistent digital memory so that we don't lose our history. So the takeaway, the final takeaway is that e-governance is a dual right and responsibility for government and for citizens. And in order to make e-governance work the right way, it's got to be based on trust. And we have to find a way to establish this. Our project is not to say we're going to establish trust, but it's to determine if there is some type of correlation between the effective use of social media and the increase of trust in government. So that's basically our bottom line. So right now we're just open for comments or questions or anything that you see there that you might have a different opinion on. I'd love to hear from you. And Ryan and I will both answer, just direct your question to either one of us. My question has to do with if it's known how many people in a city population or any metropolitan population, how many people actually use social media? You know, is the investment that a government might make in establishing and updating and also, you know, storing that social media records, do they know what part of their population uses it? You know, and that's a very good question and that will make you listen to us track again in the spring. Because the next step of this is to, we have a list of questions that we want answered. We have a form that we're using as a guide. And the information that we can find publicly, we're doing that now so we can give you stats on Facebook and Twitter and all that really. But what does that mean, right? So the next part of it, the questions that are not answered publicly that we can't glean from what we're looking at. We are now contacting representatives from the cities and arranging to interview them to ask them. All right, you have this social media initiative. If you do, you should have a policy. I don't know, but we'll ask them if they do. What are their goals for that? But more importantly, what you bring up, what are the metrics? How do you know that it's really effective? That's the next part of this. So we had, for that part, to be able to ask those questions, we had to put in a request to our institutional review board. And that went in with the interview questions that we'll be asking of the government officials. And then what we hope is that once we identify out of information from the government officials and what we found by doing our own, the two cities in Canada and in the U.S. to do case studies on. And at that point, we will be conducting citizen surveys. And the citizen surveys will be available through the websites of the cities that we select. And they will be asking the citizens their view on the use of social media by the government. And we'll be asking questions, of course. You know, how likely are you to trust that the mayor's office will, whatever it may be that they want them to do, perhaps be financially, I guess, aware of the needs of the city and be able to handle your money well. And so we're going to find that out later. But that's excellent because, no, we can't tell individual participants. You can tell numbers like I could say that was 88,000 people, right? But how do we know how many of them are different? I guess if we go through the Twitter feed, we can. We could go back. We've got all those tweets. So we could go back and try to do that. But we have not done that. What we thought would be more important was to ask the town themselves, those government officials, what they're doing, why they're doing it, how effective it is, and what kind of metrics they're using. So how can they really tell that this is working and it's worth the investment? Because you're right. That's a big problem. It's very costly. Somebody has to do that. So you're using resources, people, their time and the money you're investing in getting this up and running. So excellent question. And I guess I'm saying stay tuned for the answer. We're looking for that. Hello, Pat. And thank you very much for this awesome presentation and you too, Ryan. I just have one follow-up question. I think it's just to congratulate you. And then my first point was that I do agree with you that these Twitter, social media, they're almost diaries. And history books to your point. And I think the whole point of your research project is do we trust them? And I use the word trust in brackets. Because you've given examples of how governors have deleted social media, give given examples of how we can't almost trust the authenticity and reliability of some of the posts that are on social media. So I'm wondering from a very practical perspective as a student, from a records manager and an archivist, if it's our job to catalog these electronic records in the future and somehow preserve this data in the future, just if you don't have to answer this question, but it just might be a thought for the group is do we now have to have additional cataloging rules or additional notes in how we present this data in the future to say, oh, this is authentic. This is not trustworthy. This is not reliable. This was deleted. I'm just thinking in how we present this in the future and how, you know, it's our job to preserve and present and make things authentic and reliable. And people, if we know something is archivist and records manager, that it's not authentic. And it's not our, do we now have to like sort of put that in our cataloging notes. I'm just looking at this, this wonderful work you're doing and how we like, you know, bring it down in terms of in the day to day of the future. What will be our responsibilities? Where we now have to be trustworthy experts in our cataloging and in our preservation? I understand that's a very complex question. I don't think it's that complex because in front, if you turned out your mic, in front of my mind as you were asking that question, I see the picture of Rosemary Woods, President Nixon's secretary leaning over to grab the phone and at the same time pushing her foot down on the panel that then erased the records that were made by him that were recorded. And so there we would say we have a record, but it wasn't complete, right? And how would we know that we'd have to do some investigating. So I think we've always had these kinds of issues now they may seem more complex and more large. But to me, I try to simplify by saying if I were the government of Raleigh, the local government, I am responsible for my tweets, my Facebook pages, my posts. And therefore, I'm not responsible for the whole world. I can take a look at it. And obviously, yes. There may be some posts that are nasty or sarcastic. And that's another thing, sarcasm that we have to look at when we try to determine if posts are positive or negative because they could sound like positive but be sarcastic. But anyway, as that government, I would only be responsible for what I had put out there. Some governments don't even consider comments by others to be part of the record. So what they have to do is have a very clear records policy and understand what they believe would be the record. And then they're responsible for only capturing that and preserving that. And I think if you keep it that way. Now, if you're thinking about that whole Twitterverse, you know, Library of Congress has it and then people can start looking through it and determine for themselves. But as an archivist, if somebody gave me records just like now, I would have to determine authenticity. But I can only do that by whoever happens to be presenting them to me, right? So I would then after that for be responsible for saying they're remaining authentic. They're not going to be altered, whatever. But before that, I have to make a determination as to where I got it from. So you're right, it depends on what we're trying to look at. If somebody is looking globally at the history of the United States at this point in time, I think what they would be doing is saying, this is a picture. This is a snapshot. But unless they really got into who was responsible, who owned whatever social media was being recorded, you couldn't make that determination. Yeah, all you could do is say this is a snapshot of what was going on at this point in time. But I wouldn't worry about that. If I were responsible for records, I'd only worry about our policy, our records and doing that the right way. And I think something else important to consider is the fact that with a lot of these accounts, the citizens will call people out on stuff that they know is wrong. You have so many people who are like doing their research who are always on top of news articles and everything else is coming out. So as soon as they see something wrong, they're going to say something about it. You know, and that is a very good point. That's how a lot of actual stories get broken and that's how some incorrect information that's out there does get corrected because others will respond. And so there's a little bit of it's like Wikipedia, right? Somebody writes something that's not true when somebody else comes in there and corrects it. So there's a little bit of that correction going on. And I think even that is just such an interesting perspective on the way we've decided to communicate at this point in time. Darren, go ahead. Hi, everyone. Thanks for your presentation. So my question is about the part that Ryan presented on about the positive and negative tweets. So I think I'm pretty clear on what a positive tweet would be. Something about services, for example. But I'm not quite clear on what a negative tweet would be. And then the follow-up question is, I think I heard you say that there's some sort of program that's automatically analyzing these tweets and deciding whether they're positive or negative. And I'd just like to hear a little bit more about that. So the first part, I can definitely help you out. Dr. Freight will have to help you with the program that Dr. Chen created. I'm not completely sure on the specifics of that part myself. But for the positive and negatives, like a positive tweet, of course, is something like, thanks for filling the pot hole that was in my road. Awesome. The negative tweet, however, would be like, when people were like, why are you charging us for, why are you charging people to feed the homeless? Like, there were people who were tweeting at Raleigh's government about how horrible this program was. And that would be something that would be considered negative because it's not putting the citizens are in a good mood and it's putting the city in a bad light. Okay. What Ryan said is correct from the sentence point of view. What the researcher is, Dr. Michelle Chen, and she teaches a class in big data analysis, data analysis. She's fantastic. If you have a chance to take an elective from her, you would love it. But basically, what she does is search for particular words in those sentences. So those sentences will come off as being positive or negative as Ryan has explained. But she actually is more granular in saying the word, oh, wonderfully or wondrous is positive. If that's in a sentence likely, it would be a positive sentence. And then you would have to take a look at the context in which it's used. If you had a statement that used the word absurd, for example, or a curse, which is a strange word, or abrupt, those would be negative. They would be considered negative. And what they come from is a lexicon that has been identified and used by other researchers for other projects. So there are just a very large number, a couple, what, 1,500, maybe 2,000 positive words, 2,000 negative words, maybe even three that were used. And what Michelle did, she first thought she could get all of the tweets. We only limited the sentiment analysis is what it is. And we only limited that really to Twitter. We did not do that on the other accounts just because of time. And we did it. She did it for the 20 cities. So Twitter used to allow you to go back and get all of the tweets that you want as far as back as you wanted. They changed that in the spring, right after she did her pilot. And so she then had to find the third party site that had already saved all of the tweets from Twitter and make them available for free. The only problem there is she could only go back maybe a few months or so many tweets. And then she actually wrote her own program that went back again and scraped the tweets and brought them in again. So we actually have all of the statements, just like Ryan was explaining, but they were based on at least one word that appeared to be a word that was negative or positive. Those that were considered neither negative nor positive maybe were retweets. They were something that was done before. They really didn't have much of a message in them at all. So that's kind of how that worked out there. Michelle did a presentation on this for the Siri group. And I thought it would be really interesting if we could get her to do a similar presentation for us in the spring. I think that you'd like to hear that. But basically what she did was look for Twitter data using, you could use Topsy or another kind of application interface. And then she brought it into a data retrieval system, used that in order to get the information and parse it and validate it. So she had a lot of things going on there. And then she had to clean it up, of course. And then she had to look for things like word stemming, you know, if we say something and then we say the same word with an ING or whatever at the end. She had to look for all of that as well. And then eventually she did some charting of trends and plots for us too. So it was quite extensive. But what she used was a Topsy API. And then she created a batch script that was able to retrieve the tweets. And then she used what she called a JSON data file. And it was an XML format. So it was quite a complicated process. It wasn't something you could buy. It was something that she had to go in and actually work on. And I do see her notes here. She had 2,230 positive words and 3,905 negative words in what she calls her word bag. And that's what was looked at for scoring. So, yeah, if anybody's interested in the future, we can ask her certainly to do her presentation on her piece of it. It's fascinating to listen to her. Anybody else? Any questions? Okay. Then we're just a couple minutes after. We started a couple minutes after. So I guess we've done our hour. I want to thank you for attending. Thank you very much. And if you have any questions after this, contact Ryan or me and we'd be glad to answer them for you.