Week 19: Into to Database Systems

This week, we learned about data warehousing and business intelligence systems (DW/BI)  from the Data Warehouse Toolkit by Kimball, Ralph, and Margy Ross. These systems consider the needs of the business and then design the system with technology in mind. In organizations, record keeping and analytical decision making are important. The performance of the business can be analyzed with these tools. The information should be easily accessible and comprehensible to the business user and not in developer jargon.  It should be simple and fast for the user. The system should be labeled consistently or if they measure different things, labeled differently. The system should adapt to change and protected.

         A DW/BI manager should understand their user's job responsibilities, goals, and objectives. They should know who the best users are and find new users. They should give useful information to their users that can be trusted and is accurate. The system they make should be reliable, trustworthy, updated, and keep everyone happy.

         Dimensional modeling is the preferred technique for presenting data that is understandable to the user and fast. These are often in the third normal form (3NF) which are free of redundancies. They are often called entity-relationship diagrams (ERD or ER diagrams) who describe the relationship between tables.

         Star Schemas are dimensional models in relational database management systems. If they are multidimensional database environments, they are online analytical processing cubes (OLAP) where data is stored and indexed like performance aggregations or summary tables. They give better query performance because they are optimized and have more functions than SQL but require more computation. OLAP cubes have good security options, richer analysis, support type 2 changes, support fact tables, and ragged hierarchies. 

Week 18 CST 363: Intro to Database Systems

This week, in chapter 6 we worked with aggregate functions to do averages, sums, mins, maxes, and counts on selected rows. All of these ignore null values except for COUNT(*). The DISTINCT keyword can be used to omit duplicate values. GROUP BY is used to group a row based on an expression in ascending order and can accept multiple expressions with a comma. The HAVING clause is used to specify a search condition. The general order to use these in is SELECT, FROM, WHERE, GROUP BY, HAVING, and ORDER BY. With WHERE the search condition is applied before it calculates; in HAVING, it is after. WHERE refers to any column and can't contain aggregate functions. HAVING can only refer to a column in the SELECT clause and can contain aggregate functions. WITH ROLLUP is used to add a summary row to the end specified by the GROUP BY clause. One cannot use the DISTINCT keyword or ORDER BY WITH ROLLUP.


In chapter 7, we learned about subqueries. Subqueries are a SELECT statement in another SQL statement. Subqueries can be introduced with WHERE, HAVING, FROM and SELECT. They cannot have an ORDER BY clause but can be nested in other subqueries. Subqueries are similar to joins and can replace them. JOINS are useful when columns from different tables are used referencing an existing relationship. Subqueries are good for passing an aggregate value to the main query and with informal relationships in long, complex subqueries. The IN operator is used to give a list that is tested against the test expression and returns one column of values. NOT IN can replace an outer join. The WHERE clause is used to compare an expression with the results of a subquery. IF the keywords ANY, SOME and ALL are used, they will return a single value or a list of values. The ANY/SOME keyword is used to test if a condition is true for one or more of the values, similar to OR. A correlated subquery is executed for each row in the main query and an uncorrelated subquery is executed one time. The EXISTS operator is used to test one or more rows returned by the subquery, NOT EXISTS tests that no rows are returned. You can use an * in these to specify any column. The SELECT clause subquery needs to return one value in a correlated subquery. Using a join is a more legible option. The FROM clause returns a set that is called inline view. An alias must be used to perform calculations and can be used as column names. A complex subquery can be made but it is difficult and confusing.

WEEK 17 CST 363 Intro to Database Systems

This week, we had an introduction to SQL which included client/server systems, the relational database model, SQL and SQL based systems, and SQL statements. In Chapter 2, we learned about using the MySQL workbench, running SQL statements, and how to use the command line client. In Chapter 3, we learned about the select, where, order by, and limit clauses to process data. We used inner and outer joins to get data from two or more tables and practiced using unions. In Chapter 5, we made new rows, updated existing rows and deleted existing rows.

Initially, I had some trouble connecting to the server in chapter 2. I later found out that I needed MySQL server to do that. After reading more online and at the end of chapter 2, I downloaded the necessary programs and was able to continue with the homework.

Week 16: CST 205

A variety of subjects were learned in this class. We learned how to manipulate photos, and sounds in JES. We also learned about the software development cycle and a few basics on how to use GitHub. After that, we made a few simple games.

This class gave a good rundown of the basics of the Python language. Although it wasn't required or focused on, our team involved a lot of object-oriented programming in our code. We also worked a lot with the PyCharm IDE for its easy to use interface used by professionals and amateurs alike. I would advise a future CSIT-Online student to make sure they do the full 10+ hours of python preparation if they are not familiar with python. I would also advise someone to keep track of the new functions they have learned about and use the existing python 2.7/JES documentation for help learning about new functions. I had the most difficulty with the Labs where there wasn't as much guidance towards the end of the semester. If I got started earlier in the week, I would have done better. I also would have preferred if the TA had a group hangouts like the one we had for CST 300.

In Chapter 15, we learned about Classes and Objects. In Python, points represent a 2D space, they have parenthesis with a comma separating the coordinates. Classes are a programmer-defined type indicated by "class stuff: ". The class object can be used to create more objects. When you create a new object, that is called instantiation. The object is an instance of that class. Dot notation can be used to assign values to an instance. The elements are called attributes. Dot notation objects can be used in print statements to print the value of it. An object can be changed by reassigning one of its attributes or reassigning it with a function. Any object can be copied instead of aliasing. In a shallow copy, the object and reference are copied but not the embedded objects. In a deep copy, the object, the references, and the embedded objects are copied.

Some errors when debugging class issues will be attribute errors. This occurs when you are trying to access an attribute that isn't there. This is fixed by asking type(p); this will return the type of object. This is also fixed by using isinstance(p,Point); if this is is true, p is an instance of the class. The hasattr(p,'x') function will explain if an object p has an attribute 'x' (a string).

Week 15: CST 205

This week, we learned about lists. Lists are a sequence of values with elements or items. Lists can be changed so if there are unwanted values, they can be changed. Lists can be added together or concatenated with the '+' operator. Slice can be used on lists to remove values from the first to the last number. Append can be used to add a new element to the end of a list and extend is used to append more than one element. Pop can be used to delete an element if you know the index, and del can be used if you know the element but not the index. Split is used to break a string into words. A delimiter can be used to help split strings. 

A dictionary is like a list, it has keys and values associated with those keys. Its key-value pairs are called items. You can make an empty dictionary and add items to it. 

In the guide to identifying complexity, a few things were mentioned to do in order to increase understanding. Complex problems with sequential execution are probably not solving complex problems. Complex problems require a few self-created functions to be made and require more data variables to be tracked with test cases. They need to be broken down to be understood, then recombined and will take a long time to be completed. 

Code review is a systematic examination of computer code in order to find mistakes and improve the developer's skills. Code review ensures that the final project is good, allows practice in code review as a soft skill, allows team members to learn from each other, learning to read different styles of code, and learning to focus on details. A few tools to check out are Gerrit, Review Board, Rietveld, Codifferous. The best practices: reviewing 200-400 lines at a time at allowing yourself about an hour to do so but not more than 90 minutes. The author should annotate the source code establishing quantifiable goals and metrics to improve the process with a checklist for both parties. Then the reviewer verifies that the defects are fixed. 

Good programming practice is a good skill to learn. Start your journey with a plan which includes the data, functionalities described with pseudocode, communication between functions, and a diagram of the design of the program with the modules and how they connect. The program should be done and tested in small chunks or by lines. Programs should be indented correctly. The variable names should be clear and self-documenting in mixed case. For booleans, the word 'is' should be used to indicate boolean variables and methods. The functions should have a description of the preconditions and postcondition including the acceptable conditions where they will run. The comments should be a complete sentence with a capitalized letter on the first word unless it is the name of a variable. The header comment should be at the top of the file with the filename, author, date, version number and a description of what the file is for and what it contains as well as the course name, number, section, instructor, and assignment number. Testing data and plans should be done in the program and when testing, the boundary values should be indicated. 

Week 14: CST 205

This week, we learned about debugging techniques. There are multiple ways to do this such as glass-box, black-box, testing edge conditions, and incremental testing. In glass-box testing, every path through a program is tested as opposed to black-box testing where a range of inputs are tested. In edge conditions, the largest+1 and smallest-1 values are tested. In incremental testing, the code is tested as it is written so errors are found and dealt with as soon as they appear. When debugging with JES or other IDEs,  the debugger can be used to watch variables behavior. The error messages given and printing outputs will help as well.

In software engineering, there are two ways to engineer: top-down (eg waterfall) or bottom up. In top-down design, a list of requirements are made which describe how it should work and what tasks it needs to do. Then the hierarchical design is used to describe the functions that are needed and those are broken down into the helper functions it requires. In bottom-up design, the smallest functions are made before the larger functions.

In debugging, there are 3 types of common errors. Syntax errors are when there is a problem with the structure of the program like a missing colon. Syntax errors are indicated by "SyntaxError:".In order to avoid these, make sure not to use reserved Python keywords, have a colon after headers and loops, use straight quotes, closed multiline strings, closed operators, correct assignments in conditionals, correct indentation, and non-ASCII characters in the code. Runtime errors are made when the program is running which will describe where the error is and that was executing when it happened like an infinite recursion error. A semantic error is where there is a problem with a program that runs without producing errors but doesn't give the correct result.

Week 13: CST 205

This week, we started the sound modules.  The sounds we worked with were .wav files, which are not compressed. Sounds travel in compression waves that are created by changes in pressure. It is measured in rarefaction cycles and decibels (dB). The amplitude of the wave is the distance from 0 to the height in the y axis, a large amplitude will have a loud sound. When a full wave occurs, it is the frequency which is related to the pitch. When the frequency increases, the pitch increases.

We learned how to manipulate sound with Python. We increased the volume by getting the sample value of a sound at a specific sample then setting that sample to a larger value. This was done for all for the samples in the sound resulting in an increased volume. We then learned how to increase or decrease the volume in a given range for a sound by varying the lengths involved and varying the value.  We then learned how to combine sounds by making an empty sound and then copying the sounds into it with a for loop in the range of the entire sound clip.

In lab 8, we learned the basics of sound. We learned how to decrease the volume in a sound clip by divinding the value by half. We also learned how to change the volume of a sound by a factor by multiplying that value by a predefined factor variable. Next, we found the maximum number of samples by using the abs() function and setting the largest number it can find to max_value iteratively. Next, we set the samples to its max value using maxSample(sound).

In lab 10, we practiced strings and made a hangman game. In the warmup, we used the requestString(str) to ask for a name in a popup dialogue box. Later, a hangman class was made. This created allowed the computer to ask the user for letter guesses. If it was correct, it would evaluate the letter and add the letter to the word.