Problems of stress testing of components of billing systems

Problems of testing in general and stress testing in particular is well known among software vendors (software). It is known that, in contrast to the functional and regression testing, which focuses on testing the completeness of coverage of all branches of the algorithm of the subsystems (applications) and the backward compatibility of the functional testing services with older versions, a "headache" departments exercise testing is precisely the perspective of the approximate simulation to the reality of the load relatively few resources (both hardware and human), as well as finding bottlenecks in the functioning of the whole system.

All compounded by the fact that the process of issuing new versions or patches of errors in relation to billing systems, looks a little different than any other software. Here, the traditional process of "Staging - Development - SoftwareTesting Services- Implementation - Support comes in a very large number of relatively small components (subsystems). Since the billing system - a product of a living, constantly changing, the number of subsystems produced per day can be measured in tens.

Attempt to "integrate with" business-process software release identified a number of conflicting objectives, where it is necessary to seek a compromise.

First, each scenario is the use of billing separately (in the image and likeness, as is done at the customers) for stress testing is impossible to recreate (the number of clients is steadily growing volumes of data to be processed grows geometrically). On the other hand, make a universal stand on which to try to fully test the load all the functionality of billing and not simply because there is mutual exclusion as the algorithms within the same subsystem and the mutual exclusion of subsystems as a whole.

Second, as practice shows, the complexity of the preparations for the load testing, in contrast to the functional (regression), estimated totally different scale. By the degree of complexity of preparation for exercise testing, the software can be divided as follows:

·         "Light". Subsystems are loaded with the stream of input data (pre-filled table, a set of files, etc.). In this case, often preserved continuity tests (once prepared the input data, and then you can use them by simply changing the version of the application on the newer ones). Of course, if the format of the input data is changing (increasing), this leads to an alteration of the script. But this usually does not happen very often. Most often, the category of "light" fall batch jobs and background processes of billing systems that run directly on the server (or database or application servers).

·         "Average." Subsystems have the user interface. In this case, the client application is written "properly" - so that there is no logic in the client application is not concentrated - "light" client. In this case, the initial preparation for the stress testing can be very serious. To simulate the mass of customers, using special software tools, such as Rational Perfomance Testing services and Mercury LoadRunner, you want to create scripts from the context-related scripts written in a special language. But at the same scripts themselves are fairly simple, because contain only calls to server-side logic. In addition, in the future, the same script can be used again as a changed server-side logic (or have expanded the functional, not affecting the existing one).

·         "Heavy". Typically, these are supplements that contain the logic on the client site. Simulate the mass work with clients in such an application (especially if the tests claim accuracy) are either very difficult or impossible. For example, in the used Rational Perfomance Tester language VU Language simply does not provide floating-point operations. Ie, any such "client" the calculation involves, at least, the establishment of special libraries (DLL), allowing to make the appropriate calculations on floating point numbers, to pass a string. Thus, the tester should be in a certain sense also a good programmer! Not to mention that without the source code analysis of the simulation of such an application is simply not possible.
Thus, not every problem can be tested under load without serious "artillery preparation", and what, alas, will just have to close my eyes …

Third, a separate task is profiling loads. It is no secret that every application can work well individually. However, in view of "interprocess" correlation, the results of groupware applications, can be very unpredictable. Hence - the more accurate the profiling done loads for each of the applications, the better will stress test, and as a consequence, the conclusions drawn. There is also an application can be divided into "light" and "heavy." By the light, in terms of profiling, include those whose input flow measure (for example, the number of processed records for some period), or, if it comes to client workstations, a set of operations leading to the appearance of a certain amount of information per unit time . By "heavy" in terms of profiling, are those applications whose work poses a serious burden and, thus, leaves no "footprints" of their "life" (for example, the work «Call Center» - a group of users, giving information on telephone and, accordingly, only viewing information). Another example is a batch job (or set), process the data generated by other processes. Here, for example, there may be an avalanche effect, where due to imbalances in any of the grounds, the subsystem does not behave like in a real operation.

Currently, the group exercise testing "Peter-Service", based on the developed test methods of billing system «PETER-SERVICE BIS», was seeking answers to a rather narrow range of issues. Among the tasks we have set, such as the answer to the question of raising / lowering the productivity of key business flows billing with the upgrade version of ORACLE with the existing (9 Release 2) to the new (10 Release 2) or a comparison of hardware from different vendors at the same load scenarios. And to answer such questions today is quite simple. However, fully integrate into the business process of software production is not yet possible because of the complexity of solving the problems voiced above. Nevertheless, we are looking forward with optimism.

 Visit: Software Testing Companies

The introduction of automated software testing at the project level

Given the growing interest in test automation tools and implementation of automation projects, there is increasing demand for training and consulting in the field of quality assurance projects.

The article is devoted to recommendations on the most painless option introduction of test automation at the project level, if this trend in general is not developed in the company - software vendors.

Sure, test automation has a lot of advantages when implementing a balanced and debugged the software development process.

·         Are you interested in test automation projects, but have no experience and specialized professionals in this field?

·         How painful will be the introduction of automation in your project or department?

·         What is the effectiveness and benefits from the introduction of test automation?

This is not an exhaustive list of issues that concern the company planning to implement test automation for their projects. Before thinking about the introduction of test automation services, you must ensure that the process of quality control on your projects built, documented and works like a clock. Remember that the introduction of automation - is not a craze, and the task, designed to translate the quality control on your project to the next level. Purpose - to improve efficiency Software QA services project, rather than more headaches.

The second step is to appeal to professionals who can help you deliver the automation process in a professional manner and in a short time. Sure, you can try to develop the direction of its own, but without experienced this process is likely to result in significant time will pass by trial and error will affect a larger budget and, ultimately, it can and does discourage you wish to apply automated testing.

Modern methodologies of automated testing allows the use of consultants minimum, receive high returns from cooperation with them and successfully on their own to develop this area in your company.

When implementing projects to automate testing from scratch, we are in most cases, we recommend the development by the framework based on principles Keyword Driven approach, along with the training that will allow your team to refine their own framework and develop automated tests coverage. To date KeyWord Driven frameworks are the most technologically advanced solution in terms of price / Work / efficiency.

The advantage of this by the framework include:

·         Maximum reusable code: actually created only one script, which manages the implementation process;

·         This technology does not preclude the application of methods of Data Driven, which adds to it all the advantages of Data Driven Approach;

·         All operations are presented in the form of an Excel spreadsheet or in any other format;

·         All user interface objects are stored in external files;

·         All test scenarios (test cases) and packages start (test suites) are also described in external files (typically used Excel), allowing you to easily manage your startup;

·         The framework has the maximum flexibility: you can easily add, delete, edit existing test scripts and packages starting at this for a given task does not require additional training, only need the ability to work with the framework;

·         The system can be easily updated with new transactions or edit existing, with no need of any complex action, you should only write a new function. This allows you to easily and painlessly expand the framework itself;

·         If necessary, switch to another tool for automation, processing to be a minimum of code, test scripts will remain in the same form.

·         For many departments of software testing this solution may not be a panacea, because the results of the development framework and conduct a brief training to work with him, the requirements for qualification, covering a system Auto-test, is significantly reduced, enough skills with MS Excel.

How to build the implementation process?

As practice shows - best to carry out three steps:

·         Analysis of a set of tasks and technologies used to develop applications. According to the analysis selected the best automation tool (for a fee or free). Based on planned objectives and specific project plan for implementation and necessary process-documentation, made the necessary adjustments to the curriculum, practical exercises are based on the selected tool.

·         The second stage - the training of specialists, who must acquire the necessary theoretical knowledge of the process, organization and methods of automation, the skills of working with the tools to learn how to create test cases, combine them in a script to automate, develop basic library for building frameworks and organizations run the script.

·         The final stage - the introduction of direct knowledge in practice. At this stage, experts on the basis of the working draft, together with the consultant to plan the coverage of automated test applications, developing a framework, conducting the planned automation of the set of functional systems, develop suitable mechanisms to execute scripts and collecting results.

As a result, this should happen:

·         Fully ready infrastructure for the development of automation in the project;

·         Feature set of scripts that cover the planned amount of functional testing services with a given depth;

·         Trained, capable of independently developing automation development project;

·         Rules for the test automation;

·         Description framework and scripts for the software product on which the implementation took place.

Duration of work on such a project may (but need not) meet the following framework:

·         Analysis of problems in the introduction: "1 - 2 weeks;

·         Education (depending on the program): ~ 4 - 10 working days;

·         The introduction of an active project (depending on the amount of work scheduled): ~ 3 weeks - 1.5 months.

Outsourcing software development

Now, one of the most popular term in the business is “outsourcing“. Specialized software testing companies are now bought a variety of projects ranging from developing information technology strategy, completing the development of applications and this is due to certain advantages.

At first, giving a solution of their problems to companies that specialize in certain kind of activity (eg, application development), we can improve the quality and reliability of the solution of these problems, as well as the predictable result. Secondly, outsourcing of non-core activities will not divert its own staff for activities not relevant to their professional aspirations.

Third, a subcontractor who specializes in certain jobs, has extensive experience and has packaged solutions for common problems, which reduces the cost and accelerate their solution. Fourth, it is a company specializing in one form or another IT services, have the most advanced technologies, since they use these technologies is a key success factor. Finally, the outsourcing of application development will not have the IT department staff of developers, thus avoiding the problems associated with managing them. 

Of course, among managers and owners of companies have already appeared the first proponents of the IT-outsourcing, and is used primarily by outsourcing software development due to high cost of their own team.
Nevertheless, the outsourcing of software development has some specific characteristics, while they differ significantly in cases where the client company is not engaged in software development (software), and when the client company itself specializes in software development.

However, despite the attractiveness of outsourcing software development costs, however, aware of and possible negative consequences of the decision to transfer work on the development of software to another company. First, giving to the side of software development, the company will inevitably trust another company a certain percentage of their secrets, exposing themselves to the risk of leakage of confidential data. Sometimes it turns out that the organization of outsourcing must take extra effort to hide from the partner in charge of developing, information about the full application architecture and features of key business processes.

Secondly, there is another problem faced by customers of outsourcing software development. It is the need to invest in dive staff artist at the specifics of their business processes. Starting a project with the use of outsourcing, we have to expend some effort to ensure that partner is able to perform assigned tasks, for example, provide descriptions of business processes, and sometimes straighten on training courses on this or other technologies (particularly, it concerns the specific branch of technology ).

Ultimately, it may be that such an implicit training partner costs for the client company is expensive, and more profitable it would be these assets to invest in their employees. You have to understand that IT employees - companies are learning from project to project, and for the money the customer and the staff is not your company.

However, some of these risks can be reduced through appropriate administrative, legal and organizational measures (such as the conclusion of Executive confidentiality agreement and the requirement for a performer of a trade secret with a reference on reference to a commercial secret detention contract work). However, the conflict with the performer can lead to disastrous consequences if you do not calculate the risks of a possible change of executor and not to define the activities or ways of doing the work prevents it.

How to Report Bugs Effectively

Anyone who wrote a program for public use, has received at least one bad bug report. Messages that are not talked about anything ("It does not work"); messages that did not make sense, the messages that were not given sufficient information, the messages that were given incorrect information. 

Reports of problems that turn out user error, reports about problems that turn a defect in someone else’s program, reports of problems that turn network failures. 

Learn more about software testing services

There is reason to believe the work of technical support, which is disgusting to do, and the reason - bad error messages. However, not all error messages are repulsive: I support the free software when you do not earn a living and sometimes I get a wonderful, clear, informative posts.

In this essay I will attempt to articulate what makes a good error message. Ideally I would like to see everything in the world to read this essay before you tell anyone about the error. Of course, I would like to see everyone who reports an error to me, read it.

In short, the purpose of error messages - to allow the programmer to see ourselves as the program fails. You can either show it in person, or to give precise and detailed instructions on how to make the program broke down. If they can make it fail, they will try to gather additional information until yet know the cause. If they can not make it fail, they should ask you to collect this information.

In the error messages, try to clearly define what is the actual facts ("I was at the computer and this happened"), and that - assumptions ("I think the problem may be in this"). Lower the assumption, if you wish, but do not immerse the facts.

When you report an error, you do it because you want that the error was corrected. It makes no sense to blame the programmers, or knowingly assist them: this may be their fault and your problem and you can be angry at them, and you may be right that get angry at them, but will be fixed faster if you help them by providing all the information they need. Also remember that if the program is free, the author gives it to you out of kindness, so if too many people are too rough with him, he may cease to be good.

"This is not working"

Believe me - the programmers have some rudiments of intelligence: if the program actually does not work, they probably would have noticed it. And since they have not noticed, they should work. So, either you’re doing something wrong as they are, or your system is different from them. They need information; supply this information - this is the purpose of the error message. More information is almost always better than less.

Many programs, particularly free ones, publish lists of known bugs. If you can find a list of known bugs, it is worth to read it to see if the error you have just found a well-known or not. If it is already known, probably not worth it to report, but if you think you have more information than the error message in the list, you can still connect to the programmer. They will be easier to correct a mistake if you can give them information that they already have.

In this essay, many of the rules. None of them is not absolute. Different programmers prefer different ways of reporting errors. If the program comes with its own set of rules error messages, read them. If the rules that come with the program consistent with the rules in this essay, follow those that come with the program!

If you do not report an error, but just ask for help in using the program, you should tell us where you are looking for the answer to your question. ("I looked in chapter 4 and section 5.2, but could not find anything that would tell me if this is possible) This will allow the programmer to find out where people expect to find the answer, so he can make the documentation more user-friendly.

"Show me"

One of the best ways that you can report a bug - it’s demonstrate its programmers. Put them in front of your computer, run the program and show what is going wrong. Let them see how you turn the machine to see how running a program to see how you interact with it and see how that program is doing in response to your input.

They know the program inside out. They know which parts they trust and what parts might be defective. They intuitively know what to look for. By the time the program will do something obviously wrong, they can already notice something subtle wrong and this may give them a clue. They can understand everything that the computer does during the test automation services run, and they can draw from this are important to them part.

This may not be enough. They may decide they need more information and ask you to show them the same thing again. They may ask you to tell the startup procedures so that they can reproduce it for yourself as many times as they want. They may try to change the procedure several times to see whether the problem occurs in only one case of a family of related cases. If you’re unlucky, they may need to spend a couple of hours with a set of developer tools and start to really understand. But most importantly - to make the programmer looked at the computer when it works properly. When they see taking place before their eyes a mistake, they will be able to take it and try to fix it.

"Show me how to show yourself"

This is the era of the Internet. This is the era of global communication. This is the era in which I can send the program to someone in Russia at the touch of a button, and he can send me comments as easy. But if he has a problem with my program, he can not do it so I stood in front of his computer when it crashes. "Show me" well, when it can be done, but often this is impossible.

If you need to report bugs to programmers who can not attend personally, the purpose of exercise - to enable them to reproduce the problem. You want the programmer has launched its own copy of the program, did the same thing and broke it the same way. When they see as there is in front of them, they can deal with it.

Thus, tell them exactly what you’re doing. If this is a graphical program, tell me what buttons and in what order you pressed. If you run a program by typing the command, show them precisely what command you typed. Wherever possible, provide a verbatim transcript of dialogue, showing what commands you typed, and that the computer gives you the answer.

Give the programmer all the input data, which you might think. If the program reads the file, you will probably need to send a copy of the file. If the program communicates with another computer on your network, you probably will not be able to send a copy of this computer, but you can at least say what is a type of computer, and (if you can) what software it is running.

"Works for me. So what’s wrong? "

If you give the programmer a long list of input and action, and they have launched their own copy of the program and nothing wrong happened, it means that you have not given them enough information. Perhaps the fault does not happen on every computer, their system and yours may be something different. Perhaps you do not understand what the program should do, and you’re both looking at exactly the same conclusion and think that it’s wrong, but they think it is correct.

Thus, also describe what happened. Describe exactly what you saw. Describe why you think that something that you saw is wrong; better describe exactly what you’d expect to see. If you say "and then she did it wrong, you omit important information

If you see an error message, tell the programmer to precisely and accurately what it was for the message. This is important! At this stage, programmers do not try to fix the problem: they are just trying to find her. They need to know what went wrong, and these error messages - the best way to describe it. Record the error messages if you have no easier way to remember them, but do not tell that the program generated an error message with no description of what it was for the message.

Especially if the error message contains a number, let the programmers know them. Do not assume that they do not make sense just because you can not see it. Numbers contain all kinds of information that can be read by programmers, and they often contain key information. The numbers contained in the messages because the computer is unable to report the error in words, but doing the best that can be done to provide you with important information.

At this stage, programmers effectively do the work of a detective. They do not know what happened, and they alone can not get close enough to see how it happens, so they are looking for clues that this prompt. Error messages, incomprehensible strings of numbers, and even unexplained delays are just as important as fingerprints at the crime scene. Keep them out!

If you use Unix, the program can produce a core dump (core dump). Core dumps - an important source of evidence, so do not throw them away. C on the other hand, most programmers do not like getting a giant dump files via email without warning, so ask before you send them to someone else. Also, keep in mind that the dump contains a record of all states of the program: any "secrets" (probably the program contains a private message or has to deal with sensitive data) may be contained in the dumps.

"Then I tried it…"

There are many things you can do when an error occurs. Many of them will make the problem worse. My friend at school deleted by mistake all my files Word, and before you call a knowledgeable person for help, she reset the Word, and then tried to run the derangement. None of this helped to restore the files, and this is mixed a disc to such an extent that no file recovery program in the world could not recover anything. If she had just left it as is, she had a chance.

Users like this are like the mongoose cornered: leaning back against the wall and staring death in the face, he vehemently attacked, because doing something would be better than doing nothing. It is not well suited to the type of problems that happen with your computer.

Instead of being a mongoose, be an antelope. When an antelope is facing something unexpected or frightening, it freezes. She stands perfectly still and tries not to attract attention, while she stands, she thinks and works out the best solution. (If antelopes had technical support line, they would call back at this moment.) Then, when it decides what can be done more safely, it does.

When something goes wrong, immediately stop doing whatever it was. Do not touch any buttons at all. Look at the screen, notice all the unusual and remember or write it down. Then, perhaps, begin pressing «OK» or "Cancel", depending on what appears to be safer. Try to develop a reflex - if your computer does something unexpected - freeze.

If you have coped with the release of a problem, either by closing the program or reboot your computer, it would be good to try to make sure that this problem occurred again. Programmers like problems that they can reproduce more than once. Happy programmers fix bugs faster and more efficiently.

"I think the tachyon modulation must be badly polarized"

Not only non-programmers write bad bug. Some of the worst mistakes I’ve seen are written by programmers and even good progammistami.

I once worked with another programmer who finds errors in your code and tried to fix them. Also, quite often he discovered the mistake he could not fix it and called me for help. I asked, "What happened?" He replied by telling me your opinion about what should be corrected.

This works well when his opinion was correct. This meant that he had already done some work and we can finish it together. It was helpful and efficient.

But quite often he was wrong. We worked for some time, trying to figure out why some particular part of the program produced incorrect data, and in the end, discovers that she has not done that for half an hour, we explored an excellent piece of code, but the real problem was somewhere else.

I am confident that with the doctor he would not have done. "Doctor, I need a recipe Gidroyoyodina." People know that it is not necessary to speak to the doctor: they say the symptoms, its discomfort, pain, rash and fever, and you let the doctor make a diagnosis that is the problem and what to do with it. Otherwise, the doctor declares you are a hypochondriac or crazy, and it will be correct.

It’s the same with programmers. Sometimes it is useful to inform their own diagnosis, but has always set out for symptoms. Diagnosis - is an optional extra and not an alternative to the provision of symptoms. Equally, make code changes to fix the problem is a useful addition to the error message, but no adequate substitute for it.

If a programmer asks you for additional information not invent it! One day someone told me about the error and I asked him to try the command, about which I knew she was not working. The reason that I asked him - I wanted to know which of the two error messages it displays. Knowing which email program generated - was key. He did not try to do it, he just wrote me, "No, it will not work" It took awhile to convince him to try.

Excellent that you have the intelligence to help the programmer. Even if your deductions are wrong, programmers will thank you for what you have at least tried to make their lives easier. But please also symptoms, and then instead you can make their lives more difficult.

"It’s funny, it did so a moment ago"

Say "intermittent fault" to any programmer and see how pogrustneet his face. The easy problems are those for which the play is enough to perform a simple sequence of actions. The programmer can repeat these steps in a well-observed test conditions and detailed look at what happened. Too many problems so do not do: it is programs that sboyat once a week or very rarely, or never sboyat when you’re trying to do it in front of a computer programmer, but always sboyat when you have a suitable deadline for the delivery of the work.

Most unstable failures are not truly stable. Most of them have some logic. Some might occur when the memory ends, some may occur when another program tries to modify a critical file at the wrong time, and some can only occur in the first part of each hour! (I actually saw this.)

Also, if you can reproduce the error, and the programmer can not, it may be because your computer and his computer into something different and this difference leads to an error. Once I had a program that is folded into a small ball in the upper left corner of the screen and sat there and sulked. But she did it only at a resolution of 800×600; everything was fine on my 1024×768 monitor.

Programmer wants to know everything that you can find out about the problem. For example, try on another machine. Try two or three times and see how often it fails. If an error occurs when you are doing serious work, but does not occur when you are trying to demonstrate the cause could be a great time to start or large files. Try to remember as many details of what you did with the program when it zasboila and if you see any patterns, mark them. Anything you can tell, can help. Even if it’s only assumptions (such as "There is a tendency to the fact that it falls more often when running Emacs»), it can not provide direct clues to finding the cause of the problem, but it can help the programmer reproduce it.

Most importantly, the programmer wants to make sure whether he has to deal with this unstable failure or a failure, characteristic of the machine. He wants to know many details about your computer, so that can make a conclusion about how it differs from his computer. Many of these parts depends on the particular program, it’s one thing you should definitely be ready to announce - the version number. Version number, version number, operating system and, possibly, the version numbers of other programs related to the problem.

"Then I loaded the CD into your Windows…"

It is essential that an error had been written clearly. If a programmer can not understand what you meant, you might as well have nothing to say.

I get error messages from all over the world. Many of them are from people for whom English is not native, and many of them apologizing for his poor English. Generally speaking, error messages, with apologies for bad English is actually very clear and helpful. Most ambiguous messages coming from the people for whom English is native, who believe that I understand them, even if they do not make any effort to be clear or accurate.

·         Be specific. If you can do something in two ways, specify how you used. "I chose the Load" might mean "I clicked on the button Download" or "I pressed Alt + W". Tell me what you did. Sometimes it matters.

·         Be verbose. It is better to give more information than less. If you say too much, the programmer can ignore some parts. If you say too little, he should go back and ask more questions. One of the error messages that I received, consisted of one sentence. Every time I asked for more information, the reporter said to me in one sentence. Obtaining a useful amount of information took me a few weeks, as was adding each time one small suggestion.

·         Be careful with pronouns. Do not use words like "this" or "box" when it’s unclear what they mean. Consider this: "I launched the application Foo. It kicked up a warning window. I tried to close it, and it fell. " It is unclear what the user tried to close it. Did he close the window with a warning or an application Foo entirely? This is a big difference. Instead, you can say "I launched the application Foo, which kicked up a warning window. I tried to close the warning window, and the application Foo fell. It is longer and with repetitions, but also clearer and harder to misunderstand.

·         Read what you wrote. Themselves, read the message and see whether you consider yourself, it is clear. If you bring a sequence of actions leading to the crash, try it yourself to make sure that you have not missed any step.

Summary

·         The first task of the error message - let the programmer see the failure with their own eyes. If you can not be with him, to reproduce the crash in front of a programmer, give detailed instructions so that he could reproduce the crash itself.

·         If the first task fails and the programmer can not see the crash itself, the second problem is the error message - to describe what went wrong.

·         Describe everything in detail. Identify what you saw. Also determine what you’d expect to see. Record the error messages, especially if they have the numbers.

·         If your computer does something unexpected, freeze. Do not do anything as long as you do not calm down and not do anything that you think might be dangerous.

·         Of course, try to diagnose the fault, if you think you can do it, but even in this case, you should also report symptoms.

·         Be prepared to provide additional information if needed to the programmer. He would not ask if it was not he needs. He is not intentionally annoying (inconvenient) Let the version numbers will be at your fingertips, because they probably need it.

·         Write clearly. Say what you have in mind and make sure that it can not be interpreted correctly.

·         First of all, be specific. Programmers like precision.

Click here to know more here : Software Testing Companies

What is software quality?

The main problem in quality management is the fact that the definition of quality is too vague and ambiguous. This is due to the fact that usually the term quality misunderstood. Such confusion may be due to several reasons …

Try to answer the following questions:

• What is software quality?
• Popular opinion on the quality
• Professional approach to quality

Read more about software testing services click here

What is software quality?

In our first issue, we try to give a definition of quality and Software Qa.

The main problem in quality management is the fact that the definition of quality is too vague and ambiguous. This is due to the fact that usually the term quality misunderstood. Such confusion may be due to several reasons.

First, the quality is not a single idea or concept is more diverse and multidimensional concept.

Second, for any concepts and definitions, there are several levels of abstraction, for example, when people talk about as one of the means by this is too broad and vague sense, while another may refer to a specific definition and meaning.

Third, the term quality is an integral part of our everyday communication, but the common and professional use may be very different.

Popular opinion on the quality

Conventional wisdom about the quality is such that it is something intangible and intangible - of it may be controversy and debate, it is possible to criticize and praise, but to weigh and measure the quality of the impossible. Such expressions as "good quality" and "poor quality" are a clear example of how people talk about something uncertain for them, that they can not clearly describe and define. 

This opinion reflects the fact that people perceive and interpret quality in different ways. It is understood that quality can not be controlled and managed, and the more it can not be quantitatively measurable. Such a view clearly contrasts with the professional approach to quality management - quality is a well-defined quantity that can be measured and controlled, it can be managed and improved.

Other popular opinion that the quality is closely connected with a luxury first-class and refined taste. Dear, thoroughly thought out and more technically sophisticated product is considered as a guarantee of high quality than cheaper counterparts. Following this logic, Cadillac - is a quality car, and Chevrolet is not, despite the reliability and the number of breakdowns, or HI-FI system is a quality system, as an ordinary radio - no. Under this approach, the quality is limited to certain class of high-value products with sophisticated functionality and class products. Simply put, it is hardly inexpensive product will be classified as a quality product.

Professional approach to quality

Unfortunately, such an uncertain and vague ideas can not be used to improve functional testing services. Consequently, it is necessary to give a clear and easy to work definition. In 1979, Crosby has defined quality as "compliance» ("conformance to requirements"), and Juran and Gryna in 1970 to define quality as "fitness for use» ("fitness for use"). These two definitions are closely linked and are in excellent agreement, as we shall see later.

"Compliance" suggests that the requirements should be clearly defined so that they can not be understood and interpreted correctly. Later, at the design stage, made regular measurements of the developed product, to determine compliance. 

Any discrepancies should be considered a defect - the lack of quality. For example, the specification of a particular model of radio stations may require ability to make a certain frequency of radio waves at a distance of more than 30 kilometers from the source of broadcasting. 

If the station is unable to fulfill this requirement, it does not meet quality requirements and should be declared unfit and defective. Based on the same principles, if Cadillac meets all the requirements to machines Cadillac, then it’s a quality machine.

If Chevrolet meets all the requirements to machines Chevrolet, therefore, it is also a quality machine. These two machines can be quite different in style, speed and efficiency, but if both are measured by the standard sets for them, then they both will be high-quality machines.

The definition of "fitness for use" takes into account the requirements and expectations of end users of the product, which expect that the product or service provided will be convenient for their needs. However, different users can use the product differently, this means that the product must have the most diverse use cases. By definition, Juran each use case is a characteristic of quality and they can be classified into categories as parameters usability.

These two definitions of quality ("compliance" and "fitness for use") is essentially the same. The difference is that the version of "fitness for use" refers to the role requirements and expectations of the customer. The role of the customer associated with the quality, can never be overstated. From the perspective of the customer, product quality, which he acquired, consists of many different factors, such as: price, performance, reliability, etc.

Only your customer can tell you about the quality because it’s the only thing he really buys. The customer does not buy the product. He buys your assurance that all of his expectations for the product will be realized.

Conclusions

Let’s try again to give a definition of quality from the perspective of the customer or user of the product.

Quality - it’s suitability for use. Does the product is what I need, whether it makes my job if I can use it as I prefer.

And now look at the developer’s perspective.

Quality - is the compliance requirements specified and collected if the product does everything that is stated in the requirements.

The problem is that the specified requirements and collected is usually only part of the real requirements and expectations of the customer. Where is the correct definition of quality?

The quality of this correspondence is the real requirements, explicit and implicit. Very often implicit requirements are so obvious to the customer or the user that he does not even suggest that they are unknown to the developers. For example, let us return to our cars - the customer can describe in detail the requirements for the design parameters of the engine, interior design, exterior colors, but never point out that the tires should be round, and the windshield - transparent.

The customer will be satisfied only if the purchased product will fully meet its real and vital needs, as specified, or not.

Testing Strategy

Managers of teams testing and lead tester often develop than necessary working documents and artifacts, documents a higher level, describing the general approaches to system testing and development of the software testing services process in the project. On one of these documents, artifacts, and will be discussed below.

Many of us have been developing a testing strategy, particularly often, such artifacts are interested in the customers of major projects, the development period which exceeds one year. Let’s try to clarify the concept of testing strategy and to answer some questions dismantling the few examples in practice.

Terminology

STRATEGY - the art of leadership, a general plan of this work, based on the current reality at the present stage of development.

There is another option, with a military slant:

The science of warfare, the art of warfare. The overall plan of warfare, military operations.

I cite two definitions, although the first to accurately describe what actually we are dealing with. And although our everyday lives can be seen as waging war for the quality of the product, as we shall see anything "military" in the testing strategy is just not.

Definition

Testing Strategy - a plan of work on testing the system or module-specific functionality and dependencies with other components of systems and platforms. The strategy defines the types of tests that need to perform for a given functional system includes a description of the approaches in terms of testing purposes, and may specify or describe the requirements necessary to conduct the test tools and infrastructure.

It turns a little scary? Let’s try to split into more detailed parts using, for example, the breakdown of the issues that the strategy meets the test.

The strategy responds to the questions:

·         How, how testing will answer that this functionality works?

·         What should be done and what use of the tools for achieving the goals of testing?

·         When a specific functionality will be tested and therefore when to expect the results?

As an additional problem to be solved in the process of understanding the strategy of testing, we can consider the problem of minimizing the cost of testing. Below is an example we analyze a specific case, but now just confine ourselves to stating the fact.

We see that the testing strategy, as an artifact, fits well into the test plan. Templates, test plans offered by different methodologies, often just one of the sections include a description of a testing strategy or include a description of the strategy in points of the plan is responsible for testing of specific parts of the functional. For example, the pattern Rational Unified Process defines testing strategy as the biggest section of the test plan.

That the ideologically more correct: to provide a strategy for the entire project, or to develop specific approaches for each area of ??testing depends on the project. In modern distributed systems with several types of jobs and system agents that also act as users of the system, allocate logical strategy for each set of functional test: the module or subsystem. For simpler applications you can offer testing of all applications on a single strategy.

The customer often wants to control the testing process and see the understanding of the problem of testing for implementing the project. For him, testing strategy is less detailed document, the vision (vision) of how the system will be tested during development.

Practice

In order to move from theory to practical application, try to develop a testing strategy for the two different software solutions: desktop applications, working in single user mode and client-server system of average complexity.

Parsed following examples use a simplified model of the application in the context of interaction between application components and systems, as well as the internal logic of the application.

As a simple desktop application, take, for example, a new scientific calculator.

Scientific calculator

The whole set of functions that applications can be divided into two logical groups: the logic of engineering operations and interaction with the operating system (allocating and freeing resources, the use of system components interact with the system clipboard, etc.).

Of the requirements for the application to provide support for the 5 operating systems in 4 main languages ??of localization and implementation, for example, 50 engineering functions, each of which is clearly covered by one test. Tests, of course, include checking test cases, checking the boundary values, etc. In addition, the application can perform, for example, 5 functions in the interaction with the system (running an application, exit the application, save results to a file, work with clipboard, etc.).

Full coverage of the requirements specifies a set of 5 * 4 * (50 +5) = 1100 test runs. So how to perform all operations available applications for all supported operating systems and language localization.

For example, as you know, picked up not by accident. Let’s try to illustrate the testing strategy in action.

We return to the breakdown of the application functionality into logical groups. There is a set of engineering functionality is system interaction.

It is obvious that the work and the validity of the calculations, implemented within an application and do not use external components are not dependent on the localization of the system. Moreover, if the application is successfully running under a supported operating system and basic functionality for working with engineering functions work, then having a successful run all the engineering tests with the analysis of test results, you can talk about the performance of engineering functions under all supported operating systems and localization.

Thus, to guarantee the availability of engineering operations, the calculator can be run of 50 tests under the same environment.

Interaction with the system, starting, stopping the application, work with the buffer, writing the results to a file in this example, delivers a 5 tests. We have 5 versions of the operating system and 4 localization, which gives 20 combinations and 20 * 5 = 100 test runs.

Thus, for this example, the full scope of functionality is determined by 50 +100 = 150 test runs, including 50 tests of engineering functions are performed under the same configuration and 5 tests of system functions under the 1920 test environment.

Testing strategy in action

In this case, we figured out that it was necessary to test, defined the requirements for tests, conducted a simple analysis of test conditions and dependencies. In essence, defined the strategy: "how" and "what" test. Develop tests can be in parallel to develop a strategy that in any case, the design and development of test strategy is not a substitute.

Full coverage of the requirements specifies 1100 test runs. Parsing functionality and definition of the required set of testing gives the output from the strategy of testing 150 test runs.

For this particular example, the development of a testing strategy provides a direct benefit in the cost of testing at 1100/150 = 7, (3) times. As you can see, it makes sense. Try to consider steps for developing a testing strategy for more complex applications - distributed client-server system.

Distributed system

Approach in developing a testing strategy for distributed systems in much the same developing a testing strategy for a desktop calculator. For example, similar to the previous example, we need to identify the main areas that can be tested separately. To test the complex systems, also useful to distinguish not only, so to speak, the operational steps for testing (that is, what, how and where to be tested), but also to analyze the tactical steps for testing with the development of the system over time.

We return to a strategy of testing complex distributed systems. Carry out an indicative breakdown of functionality to the test area in order to understand how to plan testing and minimize costs.

Suppose that the system has a "thick client, application server and database server, which can operate on different physical platforms. The basic logic input / output validation of values ??and the construction of printed forms focus on the client, application server provides the presentation level and the necessary service logic (automatic data archiving, alerting users about abnormal situations, etc.), and the database provides the addition of direct storage certain part of the data sold, for example, in the form of packages of functions. Nothing particularly difficult to be selected and let the description does not scare some over action - we just outlined the problem for testing.

In order to not overly complicate the problem, restrict the range of test environments by fixed configurations of the database server (often in the industrial exploitation of a dedicated server or a cluster solution for databases) and application server. The client application provides the under with 2 operating systems and are rigidly fixed configuration of the operating system components (for example, on all user machines are regularly installed all the updates and service packs + a set of software on client machines is strictly controlled by the internal IT policy).

Volume problems

Try to describe the problem in terms of test scenarios, then there will be used to assess the complexity of the problem or part of the number of tests required for testing the functional.

Client: 50 tests to work with the data (input forms, data calculation, based on data stored in the dictionaries, search data, editing dictionaries, etc.), 10 tests to work with the printed forms (forming periods of sampling, the choice of types of records, print or export to a predefined list formats, etc.). Let the testing of the system is required, as in the previous case, 5 more tests. By analogy with the previous example we can get the following picture: having 2 environment, we obtain (5 +10) * 2 = 30 test runs to verify the functionality associated with the operating system (including functional printing and exporting in which to create new files in the filesystem) . We assume that the 50 test runs of implementing validation logic of the data can be performed under the same environment. Outcome - 80 test runs to test the client system.

Uniting under the example in testing the functionality of the application server and database. Let the application server implements the 20 teams for data processing and user sessions (excluding the operation of system connection pooling, compression functions transferred over the network traffic, etc.). The database server implements the 1910 system for data archiving operations, construction, usage statistics report, and several more operations. The general sense is that we have a finite set of test operations, and as the configuration defined in advance, we can talk about the final set of tests that must be done that would test the server functionality of the system. Outcome - 30 tests on the server side. Note that in this example, we do not touch the load carrying component testing: we are talking only about the functional testing services.

 Depending on the project artifacts

More interesting from the standpoint of developing a testing strategy in this example, will factor in the development of the whole system: the creation of new modules in the client application or entry into a system of new server agents.

What should I consider developing a testing strategy for complex distributed or client-server systems. One of the main factors influencing the strategy of testing (after separation of the test areas and an understanding of test problems in each area), is an analysis of the schedule the appearance of new functionality in the system, and often plan to develop designs and specifications for the modules and system components (from This depends on the task of design tests). A clear project plan, divided by tasks of planning, design, design and implementation of testing gives the manager the foundation needed to solve the problem, which raises the question of "when", ie issues related to test planning. Although the temporary assessment and evaluation of complexity of the problem logically extend beyond the definition of the strategy and are planning to direct testing, it is understood the application / system in the context of its development over the project is a strategic component of the test plan.

Thus, developing a testing strategy for small applications, and serious enough systems similar to the stage of the selection of test areas and understand the dependencies of the application functionality from external modules and components. Vary the strategy depending on the type of application under test can in part understand the development of the system over time, since large systems are often developed in several stages and are characterized by large in comparison with desktop applications, a set of functionality that requires Retesting in each new version. Understanding of what exactly should be tested as how specific functionality will be tested, that is the result of satisfying the objectives of testing is often the result of the development strategy of testing.

Conclusions

That in general gives a strategy of testing? Analysis of test tasks on components, the selection of test areas and, ultimately, a more complete understanding of the test tasks in a specific project. As we have seen in testing engineering calculator, understanding the problem allows us to separate the functionality of the application under test or system areas that can be tested independently, thereby reducing (and sometimes quite significant!) The costs of testing.

Let us return to the definition of strategy: "a general plan of work based on the current reality at this stage of development" - is applicable to the testing strategy is understanding "what" "where" (in what environment) and "when" will be tested. The answer to the question "how" we can give an analysis of system requirements and design tests. Agree that having a plan before the development of functional and can be confident enough to schedule tasks for testing, prepare the necessary data and test environment. At any given time, the head, relying on a strategy, he knows where he is and where to go next. Planning a test, the head of department or lead tester does not begin to understand the system and directly engaged in planning, resource allocation and deadlines for specific tasks.

Visit: Software testing companies