$18 Million - To re-design recovery.gov website? (financial, examples, administration)

[sanrene]

I wonder why it needs to be done in 21 days? Why don't they just save the expedited costs and use a cheaper, more realistic timeline?

Maybe it's one promise obama hopes to keep?

As a contrast, Facebook just spent $100 Million on servers alone.

I think the bid states they are somewhat constrained by the hardware technology.

[Sunny-Days-in-Florida]

Another wasted 18 million on hiding the truth from us and to let us know things are going to get better soon. Yea right!!!

The site is a complete propaghanda bull**** site full of just more Obama scams.

18 million, hummmm could have been used on saving millions of home that have been foreclosed on since Obama and his troops took control of everything.

[RD5050]

Originally Posted by sanrene

No, they are not. We'll just have to disagree.

Yes, they are.

And I'll leave it at that.

Not my article. The requirements laid in the bid. Since you have no idea what it says, your response lack credibility.

My responses to the actual project were based on my extensive knowledge of computer systems development .... and also partially from the Washington Examiner article you posted. I never claimed to know all the fine details of the requirements (since they were not in the article), except for the requirement I read which stated that the client wants to be able to track how stimulus money was being spent by the recipients.

It didn't and doesn't - hence the re-design.

That is pure speculation on your part.

That's not the point. You said the data was the most important thing, I said a robust, elegant, clean Database schema and all it's associated structures are.

When a database is being replaced by another database, the only thing important is the data itself. This data can be extracted to a flat file, and the old database can be deleted. The new developer just needs the format of the saved files, and they can then load the data into the new database using any one a number of methods. There are typically fastload type utilities that are included with the database package, which run in a batch environment, and can quickly read a flat file, and load it to a table.

The old database definitions are meaningless, unless there are plans to build the new database using the old database as a starting point.

Who's talking about losing data? That is a non-issue to this discussion.

See answer above.

In your world, I can just envision the mess you might have. You see, grasshopper, in a properly designed logical model of a database, there would be NO duplicate rows or columns with redundant data. Those are the kinds of things you eliminate when you create the logical model - it's called normalization and there is a process to it. You don't just willy-nilly "eliminate" or "cleanse" data - those data have relationships and integrity restraints that must be respected. How do you know which duplicate or redundant data is invalid? You have no idea the mess a database full of redundancies and violations of integrity constraints can have on a business. Now, there are applications that call for some degree of redundancies as a trade-off to performance - but this isn't one of them.

Your 2 sentences above tells me all I need to know about your experience.

Let me simply state that I have many years of development experience on multiple database platforms, so I do know what I am talking about.

Data cleansing is a very common process with databases, especially data warehouses. Rules are defined for the cleansing process which tells how data will be merged, modified, or deleted prior to being loaded into the database.

So it really doesn't matter how the original data was saved in recovery.gov version 1.0, or what issues the data may have.

The data can still be passed through a cleansing process, and loaded into the new recovery 2.0 database.

What book are you reading from? What? Do you have a little Access Database you play around with? It is fairly obvious you have little experience in creating and implementing, not to mention maintaining a robust, fully functional database system.

See above.

I'm not so sure you are - your "type" of design is nothing that I would put my name to however.

I have enough years of experience to know that you do whatever is required by your client in the timeframe the client gives you. The client's needs are the main concern. Not yours. If the client wants a website created in one month or less, then you give the client a website in one month or less.

The client's requirements are discussed between you and the client, and you then let the client know what you are capable of accomplishing in that one month period. Then everything (analysis, design, development, testing, implementation, production support) from that point forward are based on the timreframe you have.

And that is likely what happened with recovery.gov version 1.0

Yeah. I missed this one. This says it all, doesn't it?

If you understood that once you pay taxes, they become a part of the U.S. Treasury. This money is no longer "your" money. You are not "personally" paying for any specific government projects. You pay taxes, and that is all. The government then decides what to do with the money. The government can use it any way it sees fit ... including development of a recovery.gov 2.0 website.

And when the government decides to employ a website developer, the government (Obama) is the developer's CLIENT ... not the taxpayer. The taxpayer has absolutely no involvement with the developer whatsoever.

OK .... NOW FOR THE NEW STUFF !!!

Here is the Requirements Document for Recovery.gov 2.0

RAT Board Solicitation

On page 24 under Section 3.0 Terms, Conditions, and Reference it states the following:

Reference Sites (sites that provide functionality and features similar to those desired for the redesign

USAspending.gov - initial model for providing government spending to the public

So USAspending will be used as a model for the new recovery.gov 2.0 website

Pay special attention that throughout the document it refers to "The Government" as being the Client.

Also notice on the top of page 11:

Design, develop, build, test and implement new data structures and modify existing tables and views to support government functions and business processes.

So they may be keeping the original database (or some of it's tables), and modifying the tables as needed for version 2.0

The document lists two names on Page 38: Todd Richards and Valerie Bindel, who are apparently the government workers that Obama has chosen to head up this project.

Based on my experience with RFP's, this is a very thorough and well thought-out requirements document.

[sanrene]

My responses to the actual project were based on my extensive knowledge of computer systems development

Yes. A database full of redundant data that can just be cleansed and eliminated. You would be surprised how many times I have run into shoddy design and the type of quick-fix hackery you think is the norm.

When a database is being replaced by another database, the only thing important is the data itself.

This is a non-issue and irrelevant to the discussion. You brought up the issue of data as the most important part of a DB system and then deflected on the "losing" aspect.

Raw data by itself is useless without a properly functional DB that meets the requirements of the customer while also ensuring the integrity of the data, among other things - yours wouldn't appear to do that.

Data cleansing is a very common process with databases, especially data warehouses.

One of the instances that will have a high degree of denormalization. If that is the kind of environment you are accustomed to, your attitude is no surprise.

I have enough years of experience to know that you do whatever is required by your client in the timeframe the client gives you. The client's needs are the main concern. Not yours. If the client wants a website created in one month or less, then you give the client a website in one month or less.

Really? Tell me something I don't know.

And that is likely what happened with recovery.gov version 1.0

Pure speculation. I would have loved to been a fly on the wall - "just whip something up, get in online and don't worry that is inadequate - and spend a few million while you're at it".

And when the government decides to employ a website developer, the government (Obama) is the developer's CLIENT ... not the taxpayer. The taxpayer has absolutely no involvement with the developer whatsoever.

And you think obama does? The project is funded by the taxpayer and will be used by the taxpayer. It doesn't matter if the money comes from a general fund - it is still from the taxpayer.

Pay special attention that throughout the document it refers to "The Government" as being the Client.

We as taxpayers own the government, they are elected to represent the people, everything the government owns - land, parks, buildings, airplanes, computers is public property.

sites that provide functionality and features similar to those desired for the redesign

Just great. For $18 million they will basically duplicate an already functional, up and running system/website. What, if anything is wrong with USAspending.gov?

It will be really interesting to see if they get this up and running by the end of August - and how much in cost over runs they incur.

[RD5050]

Originally Posted by sanrene

Yes. A database full of redundant data that can just be cleansed and eliminated. You would be surprised how many times I have run into shoddy design and the type of quick-fix hackery you think is the norm.

How do we know there are any issues with recovery.gov version 1.0's data?

I personally have never looked closely at the data. Have you?

My reply about cleansing, is that even if this data does have problems, it can be dealt with using data cleansing.

Based on the exceptionally high quality of the recovery.gov version 2.0 requirements document, I would assume that recovery.gov version 1.0 was probably done equally as well, but with significantly less requirements.

So my guess, is the database used in version 1.0 (a MySQL DB) is probably of high quality as well. But that is obviously an assumption.

This is a non-issue and irrelevant to the discussion. You brought up the issue of data as the most important part of a DB system and then deflected on the "losing" aspect.

Raw data by itself is useless without a properly functional DB that meets the requirements of the customer while also ensuring the integrity of the data, among other things - yours wouldn't appear to do that.

Yours? I don't have a database.

And I am very familiar with normalization processes, and have worked on many database applications over the years.

I am also very familiar with data cleansing processes. And data cleansing is a requirement under many circumstances when you load new data into a database or a data warehouse. You can't always guarantee that the sources of external data will be in a format necessary for the database.

One of the instances that will have a high degree of denormalization. If that is the kind of environment you are accustomed to, your attitude is no surprise.

You must believe the world of data processing is always perfect. It is not.

When you work for a large company, you will have data from many sources, including 3rd party external companies who manually key data for you, and you can't always guarantee the data will be clean. That is why data cleansing routines are often required before you load data into a database.

Pure speculation. I would have loved to been a fly on the wall - "just whip something up, get in online and don't worry that is inadequate - and spend a few million while you're at it".

And what you have been saying since the very start of this thread hasn't been PURE SPECULATION on your part???

And you think obama does? The project is funded by the taxpayer and will be used by the taxpayer. It doesn't matter if the money comes from a general fund - it is still from the taxpayer.

Obama probably interacts with the developer through his assigned staff. I listed their names in my previous post.

We as taxpayers own the government, they are elected to represent the people, everything the government owns - land, parks, buildings, airplanes, computers is public property.

You can go on believing the taxpayer is somehow connected to this project ... but you'd be wrong. I can't think of any other way to put it.

The Government makes it's own decisions. It does not ask the taxpayer if it is ok to create a recovery.gov 2.0 website.

But like I said ... believe whatever you want.

Just great. For $18 million they will basically duplicate an already functional, up and running application.

Try reading the actual requirements document and tell me if they are simply planning to duplicate an existing website, or will they be creating a totally redesigned website with a lots of new and required features that don't exist anywhere else?

[sanrene]

How do we know there are any issues with recovery.gov version 1.0's data?

That was a response to your assertions about your experience - since you claimed having redundancies was really no big deal.

My reply about cleansing, is that even if this data does have problems, it can be dealt with using data cleansing.

Yes, but in your world it would continue to happen, which can be costly in time and resources to "cleanse" the data that should never have entered the database in the first place.

Yours? I don't have a database.

Your attitude - I should have been clearer.

I am also very familiar with data cleansing processes.

We were discussing duplicate rows and redundant/invalid data, which presents a challenge to the cleansing process. Cleansing is good for missing or incomplete data, not necessarily for duplicates.

You must believe the world of data processing is always perfect. It is not.

Hardly perfect. With a little forethought and adherence to good planning/design methodology, ones sanity can be so much better.

Obama probably interacts with the developer through his assigned staff.

Absolute hogwash. There are many, many layers between obama and the ones who will develop this project.

Try reading the actual requirements document and tell me if they are simply planning to duplicate an existing website, or will they be creating a totally redesigned website with a lots of new and required features that don't exist anywhere else?

Well....according to you it should be no problem to modify and restructure the existing system to incorporate the shiny new features required for this project. According to you, they can modify the DB, stick in a couple new tables, add a few reports and it would be good to go. According to you, it should be a piece a cake.

Maybe they should consider that - maybe they should say, "you know, we have this here system up and running, let's just do a partial redesign, let's save the taxpayers $15 million".

I guess that means that USAspending will be scrapped to make room for this $18 million + project.

[RD5050]

Originally Posted by sanrene

That was a response to your assertions about your experience - since you claimed having redundancies was really no big deal.

You are being ridiculous. I stated that if redundancies do exist in the present data, they can be eliminated through data cleansing.

I never said I personally would design a system which would allow new redundancies to occur.

Yes, but in your world it would continue to happen, which can be costly in time and resources to "cleanse" the data that should never have entered the database in the first place.

In my world (which happens to be the real world), you have to deal with all possibilities. And duplicate data is always a possibility if your data is coming from an outside source, such as a data entry service, that manually keys data from written documents.

We were discussing duplicate rows and redundant/invalid data, which presents a challenge to the cleansing process. Cleansing is good for missing or incomplete data, not necessarily for duplicates.

I've spent years working with data cleansing, and it handles all types of problems. It all depends on how the rules are set up within the cleansing process.

Eliminating duplicates is a very simple process. Even simple sorting utilities have the ability to eliminate duplicate records.

If you are concerned that you might not know which value to choose if you have two nearly identical rows which have one column whose values differ, you set up a rule telling it to choose the value from either the newest row or the oldest row. Whichever value makes the most sense for a given application. Or use some other decision factor, such as if one value has a higher priority than the other, then choose the one with the higher priority. The data cleansing rules are decided based on the specific application involved.

Hardly perfect. With a little forethought and adherence to good planning/design methodology, ones sanity can be so much better.

Once again, if the source of your data is from an outside company such as a data entry service, you may not have control over what comes in. You have to be ready to handle all possibilities.

Absolute hogwash. There are many, many layers between obama and the ones who will develop this project.

You are taking my comments way too literally. Obama is aware of recovery.gov, is he not? So he obviously is being kept updated on it's progress. Probably not at a low-details level, but at least at a high-level. And if something about the website doesn't sound right to him, or he has a new idea for the website, he obviously will convey it to his staff members, who will relay in down the line until it reaches the developer. So (as the client) he is indirectly communicating with the developer.

Well....according to you it should be no problem to modify and restructure the existing system to incorporate the shiny new features required for this project. According to you, they can modify the DB, stick in a couple new tables, add a few reports and it would be good to go. According to you, it should be a piece a cake.

So you are speaking for me now?

I never said any of the above.

I did say that they may incorporate the existing tables into the new website, with modifications to these tables as required for the new functions the website will perform. That is clearly stated in the recovery.gov 2.0 requirements document.

Maybe they should consider that - maybe they should say, "you know, we have this here system up and running, let's just do a partial redesign, let's save the taxpayers $15 million".

I guess that means that USAspending will be scrapped to make room for this $18 million + project.

First ... we don't know what they are planning, or how they plan to do it. The requirements document is just that, a "requirements" document, and nothing more.

It lists the new features which are required for the new version. How the developer chooses to get the job done is another subject entirely. After the developer conducts an analysis on the existing websites (recovery.gov 1.0, USAspending, and others listed in the requirements document), they can determine if any of the existing program code, database tables, ideas for screen layouts, web pages, etc. can be incorporated into the new version.

Second ... as I recall, the costs were around 9.5 million for the current year, and around 8.5 million (in total) for subsequent years through 2014.

The breakdown of these costs may include a lot more than just website development. They may include ongoing backup services, security, production support, hardware, website and database maintenance services, etc.

TRY READING THE ACTUAL REQUIREMENTS DOCUMENT ... before you continue wasting time complaining in this thread.

[Rargus147]

Hey everyone smartronix.com has a new press release. i think its legit, 18 million for a secure government website.

http://www.smartronix.com/portals/0/...ss_Release.pdf

[sanrene]

I stated that if redundancies do exist in the present data, they can be eliminated through data cleansing.

In my world (which happens to be the real world), you have to deal with all possibilities. And duplicate data is always a possibility if your data is coming from an outside source, such as a data entry service, that manually keys data from written documents.

Seems you have whittled down your argument to data coming from an outside source, but that's really not what we're talking about here. Even so, if the proper data integrity restraints are in place and the data is actually validated as it comes through, it would never enter the database in the first place. So then we are back to good planning and design.

You are taking my comments way too literally. Obama is aware of recovery.gov, is he not? So he obviously is being kept updated on it's progress. Probably not at a low-details level, but at least at a high-level. And if something about the website doesn't sound right to him, or he has a new idea for the website, he obviously will convey it to his staff members, who will relay in down the line until it reaches the developer. So (as the client) he is indirectly communicating with the developer.

There is no way he is "in the loop" on the myriad of issues and details having to do with a project of this size. Beside the fact he has no expertise in the area. Again, the taxpayer is the client (paying for it) and the end-user (using it) - obama is doing neither of those things.

I did say that they may incorporate the existing tables into the new website, with modifications to these tables as required for the new functions the website will perform. That is clearly stated in the recovery.gov 2.0 requirements document.

So then I guess they could do the same with USAspending.gov and not waste anymore of the taxpayers money - which is really the point of this thread.

TRY READING THE ACTUAL REQUIREMENTS DOCUMENT ... before you continue wasting time complaining in this thread.

Aren't you the one that said you had no need to read anything pertaining to this project?

[newtoli]

Originally Posted by Rargus147

Hey everyone smartronix.com has a new press release. i think its legit, 18 million for a secure government website.

http://www.smartronix.com/portals/0/...ss_Release.pdf

Thanks for the link. Again, the intital order is $9.5 million, not $18.

Smartronix announces it has won an initial $9.5 million task order award for the complete redesign of the Recovery.gov Web site. The task order includes the procurement, installation, configuration, security, and 24x7 operation and maintenance of a robust, secure, and highly available web infrastructure to support millions of users. A mirrored continuity of operations environment is also included in this award. If all options are exercised, the award is valued at $18 million over five years.

Again, this is not a simple "re-design".

Smartronix will be providing cyber security, infrastructure, systems engineering and data warehousing support for this award.